There’s No Wave Function? | Jacob Barandes [Part 1]

I didn’t know it was impossible. Therefore, I brought these theories closer together, anticipating that there would be a moment where I’d need to make a kind of jump to transition from one theory, classical physics, to the other, quantum physics. And the strangest thing happened. My two cliffs merged. And there was no longer a chasm.

Today, I have a special treat. I traveled to Harvard University to meet Jacob Barandes in person, the co-director of graduate studies in physics and the person who is making waves at the intersection of fundamental physics and philosophy. In his modest office, he is working on a revolutionary framework that finally explains quantum mechanics. His radical proposal suggests that there is no fundamental wave function, that Hilbert spaces aren’t real, and there is no mysterious quantum world. It’s just a mathematical convenience. It’s subservient to the theory. But there is nothing in physical reality that it describes. In this episode, we delve into the picture that solves the measurement problem, eliminates the need for multiple worlds, and finally gives a clear view of what actually happens at the smallest scales of nature. However, this clarity comes at a price. And the question is: Are physicists ready to accept quantum mechanics without magic?

Professor Jacob Barandes, welcome to Theories of Everything. I’ve been following you for a while now, stalking you online in a non-creepy way, and we've been talking off-air about physics. However, you're also the co-director of graduate studies here in physics, and you have a fascinating backstory around how you got into physics. So why don’t we start there? Why don’t we start from the beginning?

From the very beginning, my first intellectual passion was the philosophy of mind. When I was five years old, I was sitting in one of those little gathering rooms that they have in schools, you know, like my elementary school, and one of the teachers was talking loudly about something. I don’t remember exactly what they were talking about, but I remember sitting on this soft carpet, and there was a piano, you know, the piano legs look like lion’s paws. And suddenly, I became aware of my existence in a very clear and explicit way. And I was extremely puzzled, and I tried to communicate my state of puzzlement to one of the other children sitting next to me, and I utterly failed to be able to explain what I was feeling at that moment. And so that left a kind of mark on me. And from that moment on, I was very interested in questions related to, you know, what is the nature of our existence? What does it mean to be conscious? So the philosophy of mind is where things started for me. But of course, I didn’t have the language to express any of these ideas. I loved science documentaries. I grew up in New York City, and my parents would take me to the American Museum of Natural History in New York City, which is a truly wonderful and magical place. And anybody who goes to New York City should really visit it. It’s amazing. So I became really interested in science. I became interested in mathematics. And I figured that if you’re interested in science and mathematics, and you’re interested in foundational questions, that means you should go into physics, right? That’s where I figured the mathematics and these foundational questions, where they come together. So I went, you know, through school. I got to high school. I first studied physics in an academic way in high school. I loved it. It was my favorite class in high school. I still have all my old notebooks from that time. It was the only class where I was excited to go home and do the homework, and that was great. My teacher, Doug Bartel, was exceptional. He was absolutely amazing. Around that time, I also started doing weekend trips back to New York City. We moved out of the city at that point, about an hour north. And so I would take the train and go into the city on weekends. And I would take science classes in the city. Columbia University has a program called the Science Honors Program. At that time, it was run by Alan Blair, a wonderful human being who was a physics professor there and also ran this special program, the Science Honors Program. And these were classes you could go to. There’s no homework or grades or tests. You just show up on Saturday morning, and you’ll learn all sorts of amazing things. It was an amazing program. And I would also work at the American Museum of Natural History. So I started interning there when I was in high school. And I started working in the astrophysics department. I wasn’t an expert in astrophysics in high school, but I was very curious about it. And I was doing a lot of computer programming at that time. And they were interested in having me work on some of the projects that they were doing. They were working on computer graphics visualizations of the galaxy. I worked on a project called the Digital Galaxy Project, if I remember the name correctly. And that was my first opportunity to work with scientists. And I didn’t know any scientists. We didn’t have any scientists in our family. My dad’s a doctor. But research scientists, I had never met anybody like that. I didn’t know academics or professors or anything like that. So those two experiences were very important for me. And with the physics class, I decided I wanted to study science and maybe physics. But my love for the philosophy of mind was still very strong. And so when I went to college, I decided that I would try to study the brain. I loved computers. And so the idea was that I would somehow find a way to combine my interest in computers and neuroscience somehow. And in order to go in that direction, I had to take some foundational science courses, including physics, and I had to take math courses. And I really fell in love with these classes. I started studying biology in my sophomore year. And I enjoyed it, but not as much. And I have a lot of respect for people who work in biology, but it wasn’t for me.

Yes. For me, I just want to know if the reason is the same for you. For me, it’s just a plethora of terms. And I feel it’s more memorization than derivation.

Yes, I sometimes feel that way. I mean I want to clarify that I’m not a biologist. And I don’t want to, you know, make any assumptions about the way biologists think about their subject. I know that there’s an internal logic to biology.

Oh, I just mean that from the freshman level, at least from what I’ve been through, as you get older, or as you become more experienced in research, maybe it’s different. I had that experience when I took the first semester of biology, and it was microbiology, and it was very logical. Then the second semester was anatomy. And that was much more memorization. And that I had the same experience that you’re describing, it wasn’t really for me. You know, but it was hard to convince myself that I should study physics because, you know, there were all these very smart young people who were studying physics and math, and I was interested in the theoretical side, the mathematical side of physics physics. And I didn’t feel like I was good enough to keep up with people. But in my sophomore year of college, you know, I had conversations with some really wonderful people, some of my fellow college students. And basically, I decided, you know, that it’s really important to do what you’re passionate about, and then, you know, you want to be the best at something. So I’ll do, I’ll do it anyway. And so I just started taking a lot of physics and math courses. And it was great. It was an amazing experience. And some of the philosophy courses that we were required to take at that time didn’t really appeal to me. There was a heavy focus on ancient Greek philosophy and early modern philosophy. Now, I appreciate those subjects, but, you know, they weren’t as exciting to me as what I was doing in physics, at least at that time. Now, I should say that, you know, over the years, and I started exploring other areas of philosophy that weren’t part of the standard curriculum in college, analytic philosophy, philosophy of science, logic, foundations of mathematics, philosophy in physics, my views about philosophy have completely changed. But, you know, at the time, I wasn’t, I didn’t have the opportunity to see a lot of that. And I didn’t see it grow up. And so I thought, okay, I’m going to focus my attention very much on physics. So I finished my undergraduate degree, and then I went on to graduate school. And I started a PhD here at Harvard University in theoretical physics. And as I progressed through that degree, you know, I started thinking again about some of these old philosophical questions that I had. And I did my PhD in high-energy theoretical physics. And you know, I think we did some really good work. But I was actually much more excited about the philosophical questions. And by the time I finished the PhD, I knew that that was the direction I wanted to go in. So I continued to work in the department. I’m a lecturer here in the department, and I also co-direct the graduate program as I think we talked about. So, you know, I mix teaching physics and doing research and advising students. It’s been a really rewarding experience. The students here are amazing. They’re ideal, and they’re wonderful. And they’re deep. And I’ve really met the most wonderful people that anybody could ask to meet here. I’ve made lifelong friends here. I mean, it’s just been an amazing experience to be here. And I think I’ve learned a lot, much more than any of them have learned from me. But that’s how somebody who works primarily in philosophy, philosophy of physics, finds themselves embedded in a physics department. So I have a faculty appointment affiliated with the philosophy department, and I rotate teaching physics classes there as well.

Yes. But my research is primarily about the intersection of philosophy and physics. So what happened to the philosophy of mind?

That’s a good question. I’m still really interested in the philosophy of mind. I don’t think my particular set of tools is the one that I think, you know, we all come into this world with a distinctive profile of strengths and weaknesses. I’m very interested in the philosophy of mind, but I don’t think my distinctive profile of strengths and weaknesses is optimal for working in the philosophy of mind. So I read about it. I’m intensely interested in it. But I feel I’m better suited to the questions, the intersection of physics and philosophy. And that’s where I’ve spent, I’ve spent my time and my time and my focus.

You said earlier you don’t want to pursue something just to be the best at it, but you want to pursue your interests. But you’re saying here, well, my interests still lie in the philosophy of mind, but it’s not that, I’m not the sharpest tool in the shed when it comes to that. That’s not my skill set. How do you reconcile those?

That’s good. Yes. I mean we all change as we get older, right?

Yes. So I think for me it’s been about finding the balance. It’s less important to me. And I, you know, I’ve talked to students about this as well, right? It’s less important, you know, what you want to be. It’s not important to be big or famous or important. And for me, what’s most important is to do meaningful work, which I feel is meaningful, which the person feels is meaningful work we feel is meaningful, but we also feel that we’re making progress. So it’s not that I don’t want to work in the philosophy of mind because I think that would not make me famous. My lack of motivation to work in the philosophy of mind, at least right now, and this could change, is because I at least right now, I don’t feel that’s where I’ll be most productive. It’s where I feel I won’t feel I’ll do work I felt as proud of as the work I do in the philosophy of physics and philosophy of science. And that’s why primarily I focus on these areas right now.

I’d like to get you to justify the non-existence of the wave function soon. However, I want to know if there is a relationship between the philosophy of mind and quantum mechanics first.

Yes, that’s a very interesting question. And in fact, that will lead us to a very important research paper which I hope we can talk about from the early 1960s. It was a paper written by Eugene Wigner. I think the paper was called The Question of Mind and Body. The title might be slightly off. But Wigner presented a thought experiment that I think is connected in some ways to his view that there’s something that connects consciousness and the collapse of wave functions. This thought experiment actually goes back earlier. It’s not entirely clear to me who originated this thought experiment because it first appeared in Hugh Everett’s extended dissertation. Hugh Everett was a graduate student at Princeton University and was at Princeton at the same time overlapping with Eugene Wigner. And Hugh Everett originated the many-worlds interpretation of quantum mechanics and wrote this very long dissertation. It’s 137 pages. I think that’s maybe not long in the absolute sense. And in 1956-1957, he didn’t end up publishing it. He was strongly encouraged by his advisor to publish a different, shorter dissertation. But you can find a copy of this extended dissertation and it opens with this same thought experiment. So we know that this thought experiment was presented before Wigner by at least seven years before Wigner’s paper or something like that. But that wasn’t published. Wigner’s paper was published and therefore is widely attributed to him. Wigner presented this thought experiment which I’ll talk about in more detail if you’d like, because I think it gives a good motivation for why one might think there’s a problem with the way we formulate quantum mechanics at a textbook axiomatic level. So I’ll come back to that. But I’ll just say that this thought experiment was apparently connected to Wigner’s idea that consciousness is somehow connected to the collapse of wave functions. And I think that creates a potential connection between the philosophy of mind and quantum mechanics. But there are other connections. In the many-worlds interpretation, which is a formulation of quantum mechanics that Hugh Everett presented in the mid-1950s, one encounters some fundamental questions about consciousness and human identity in a universe in which a giant universal wave function splits into parallel realities, each of which contains copies of yourself. How do we make sense of this kind of picture and make it connected to the way we think about fundamental problems in the philosophy of mind? I don’t claim to have an answer to that question. I think it’s an ongoing puzzle. So I think there are some interesting connections between the philosophy of mind and quantum mechanics. And I think my general attitude has been that anytime you’re unable to answer a question about quantum mechanics because you’ve connected it somehow to the philosophy of mind, and in particular to an unanswered and perhaps unanswerable problem about the philosophy of mind, I’m very concerned.

Okay, let’s talk about the wave function, because when you talk about the collapse of the wave function, the collapse of something, what is the wave function? And why do you think there isn’t one?

That’s a great question. So let me say a little bit about some history. I think some history here might be helpful in setting things up. But before I talk about the history, let me first answer your question. I’ll give a short answer, and then we’ll talk a little bit about history, otherwise we might get lost in the weeds. There’s a common story about quantum mechanics that many students coming into the field hear. The first fundamental object in quantum mechanics is the wave function. It’s this mysterious, wavy object. It’s kind of unusual. It’s kind of like the electric field, except that its values are complex values. They’re complex numbers. They’re like ordinary real numbers plus a number times the square root of minus one. And somehow, they’re connected to probabilities somehow. And somehow, the whole point of quantum mechanics is wave functions. They’re the central player in the story. And the way you think about quantum mechanics starts from the way you think about wave functions, right? And they’re the heroes of this story. I think there are some good reasons to be a little skeptical of that picture. And I don’t think there’s much really—there’d be a lot of debate about saying—I think, you know, as you learn about quantum mechanics and you start to see that there’s a much bigger picture, you know, you start to see that the wave function in this kind of story is an oversimplification. I think that if you pressed a working physicist hard enough, they would admit that, in fact, that picture is a little bit—it’s a little bit quick, and there’s a deeper picture underneath. And so I think that one view of what I’m saying here, your question, is to take those concerns and take them to their logical conclusions, right? We ask ourselves, well, what is this theory really about? Is it really about wave functions at all? And in that regard, are wave functions the things that describe things that actually exist in the world? And this is where one lives in a kind of borderland between physics and philosophy when you ask questions like that. On the one hand, it’s about physics. It’s about, you know, what’s the right way to think, and what’s the right way to formulate, you know, what can be our best and most successful scientific theories, and certainly when it comes to numerical accuracy and precision. You know, and on the other hand, what’s really out there? What’s in the world? What exists, right? That’s the ontological word that metaphysicians use. Ontology is the study of what really exists. And let’s ask, does the wave function describe something physically existent? That’s a question. To explain a little bit about how we got here, I can say a little bit about history, but let me pause and ask, do you have follow-up questions about this? I want to be careful, you know, to make sure we all have time for this.

Certainly, certainly. So when you say there’s the standard picture of the wave function, you said, well, what people learn is that there’s a wave function, and it’s associated with complex numbers, and it’s also associated with probability in some way. Is that the picture you mean? Or are you referring to the picture that represents the wave function as something in physical space and it’s of something?

Yes. So when I say that, what I mean is, when people say the wave function, when physicists talk about the wave function, it’s the wave function of something, the wave function of the electron, the wave function of so-and-so. So what’s that picture that you’re about to shatter specifically?

Yes, that’s an excellent question. I think, you know, the problem here is that it contains a little bit of both, right? The wave function plays a functional role in quantum mechanics. That is, it does certain things for you theoretically. And we study its change with time. There’s this famous equation called the Schrödinger equation, which roughly says that when the system is left to itself, it’s not disturbed, it’s not exchanging information with any other systems. Then it has a wave function that changes with time in a smooth way. And this smooth process is described by a differential equation, which is roughly an equation that just tells you what the wave function is at every infinitesimally later moment, given what it was at the previous moment. And that equation is known as the Schrödinger equation. So we plug in the initial wave function, and we use the Schrödinger equation to find out what the wave function will be, how it will behave as time goes on. And then there are rules for taking the wave function and calculating predictions. So that’s uncontroversial, and I certainly agree with that picture, where we should use wave functions for this purpose. But I think it’s easy to let that picture lead one to believe that wave functions describe physical things that exist in the world. And you know, when I think students arrive at a quantum mechanics class for the first time without having studied the theory, they might think that, well, every particle has some wave function, and wave functions are kind of like, they’re kind of like waves. And you know, empty space is kind of like an ocean, and all the particles are like little waves that move in the ocean. And that’s actually not the picture that we get even from the textbook theory of quantum mechanics. In textbook theory, the wave function is an assignment of a complex number to every possible configuration of your system. If you’re talking about one particle, nothing else, then every possible configuration is every possible location the particle could be. And each location is distinguished by three numbers. They’re the x, y, and z coordinates, if you like. And so the wave function takes these three coordinates and assigns to them a complex number, and then we’re supposed to do a mathematical operation with that complex number, and the probability comes out. More precisely, the probability density, but the probability per unit volume that measurement will find the particle at that spot. So you can think of the wave function as an assignment of a special complex number to every point in three-dimensional physical space, where you do this operation, which is called mod-squaring, and you do this operation, and then it tells you the probability, roughly, with which measurement will find a particle at that location. So it’s very easy to think that the wave function is like a field, like the electric field, in three-dimensional space. But if you have two particles instead of one, the probability space is much more complicated. We call this probability space the configuration space. Because every possible configuration of a two-particle system requires six numbers, not three. You need to know x, y, and z of the first particle, and you need to know x, y, and z of the second particle. So the wave function assigns a complex number to a point in six-dimensional space. Six-dimensional because you need six numbers to specify its points. And that means that the wave function of two particles is a function, we say, its domain is six-dimensional space. Six-dimensional space is not three-dimensional physical space. If you have three particles, it’s nine-dimensional space. If you have ten particles, it’s thirty-dimensional space. And in fact, our universe doesn’t have a well-defined number of particles at all. Our leading physical model of the universe, at least for the non-gravitational parts of the universe, is the standard model, which relies on a bunch of models known as quantum field theories. And in quantum field theories, particles are emergent excitations of these kind of delocalized entities called quantum fields. And the number of particles can change from moment to moment. It’s not always well-defined. So it’s not even clear how to think of wave functions living in anything like physical space in a universe as we know it. So I don’t mean to say that we’re teaching our students incorrectly. We’re not. I mean the students will learn this as they progress in their physics trajectories. But I think many people outside or new students who haven’t yet begun their own journey in physics have a certain idea about what wave functions are which is actually quite different from the way we use them in practice. I don’t know if that clarifies what you were asking.

Okay, you have a lecture on how Hilbert spaces aren’t real as well. I’d like you to talk about that. Now, I understand you said more precisely that Hilbert spaces are redundant in the same way that measurement symmetries are redundant or that measurement transformations are redundant. So at this point, if it’s alright, let me talk a little bit about history, if that’s alright.

So where does quantum theory begin? The lightning rod, like a grand tour of quantum theory. There are these actual physical things called black bodies. They’re rooms that are heated up. And you know, there’s a little hole in them. I mean you don’t actually drill a hole. It’s more complicated. I’m very much a theorist, and my picture of experimental physics is that it should be better. And I apologize in advance to any experimentalist who’s listening. But these were actual systems. People had actually built these things. They were rooms. They would heat them up. And then they would have a little hole in the room through which they could see the radiation coming out of them. And they could put that radiation through experimental apparatuses that could show them how strong the radiation is as a function of wavelength. We all know that different wavelengths of light correspond to different colors in the visible range. And if you go outside the visible range, you have very long wavelengths infrared or, you know, microwave radiation or radio waves. And at very short wavelengths, you have x-rays and so on, gamma rays going out, and ultraviolet, and gamma rays going out to very short wavelengths. And you could ask yourself, if I plot how strong the radiation is as a function of wavelength, what kind of pattern do I expect to see? And it was hard to explain the pattern that was revealed in experiments on the basis of a first-principles theory. A very important physicist, Max Planck, in 1900, discovered a way to generate a theoretical prediction for what the black body radiation curve should look like. And it agreed with experiments. But in order to get there, he had to kind of fudge his formulas. He had to introduce a fudge factor. He had to assume that the radiation in this room could only occur in quantized amounts. That means that there were different wavelengths you could produce, but each wavelength could only be excited in discrete steps. This led to

The quantum hypothesis. This was his quantum hypothesis. There was a kind of parameter that tells you how much you want to implement these steps. Today, we might call it the regulator. And this parameter, we now call it h, little h. We call it Planck's constant. Planck introduced it, and I think his idea at some point was that he wanted to introduce an intermediate step in the calculation, and once he calculated everything, he would send this parameter, clearly in his mind, to zero. But whenever he tried to send it to zero, he got wrong results. He realized that this parameter was not zero, but it was very small. It was, today, it has a value of the order of 10 to the minus 34, and 10 to the minus 33 in conventional units, and he couldn't get rid of it. And so people had to accept that there was this fundamental distinction in nature that he couldn't explain.

Then over the next 22 or 23 years, the period from 1900 to 1922, 1923, the physicists who were working on these questions were living in a time we now call old quantum theory, a model we call old quantum theory. Old quantum theory was based on a mixture of physical pictures of particles moving in space, in orbits, as in atoms, and then a set of ad hoc formulas and rules that people didn't quite understand that seemed to capture some of the observations that were coming out of experiment. But it was a very murky time. There was nothing like a fundamental theory from which this whole picture emerged. And the formulas weren't perfect, and they didn't work quite right.

In 1913, Bohr proposed, as in Niels Bohr proposed his famous model of the hydrogen atom. This is a model where you have the nucleus, and for the hydrogen atom it's just a proton, a positively charged proton in the middle, and an electron orbiting it. And by leveraging this kind of ad hoc formulas that were characteristic of the time, Bohr argued that the electron could only be in certain specific orbits, and when it jumps between them, its energy changes in discrete steps, and whenever its energy changes, it either absorbs or emits radiation in discrete amounts. And with this, he was able to explain the specific colors of radiation that we can see emitted from excited hydrogen atoms, or absorbed by hydrogen atoms, which are called line spectra. And we also found that other atoms have line spectra as well. Helium, it's well known, was discovered because we looked at the spectral lines of the sun, and after we accounted for all the spectral lines that we knew, there were still more lines that we hadn't accounted for, and so people guessed that there was a new element. They called it helium after Helios the sun, and eventually we found helium on Earth, but it was first discovered in the sun. And clearly, this was something very important, and Bohr ended up winning the Nobel Prize in 1922 for this Bohr model.

But by about 1922, at least by that time, people were becoming extremely skeptical, because people couldn't quite capture these clear pictures of particles moving in definite ways, particles interacting with fields like the electric and magnetic field, which were well understood by this point. They couldn't capture this picture of the world, this ontology, this picture of what was physically there. They couldn't find the laws that when combined with this picture gave you the right predictions, that gave you sufficient empirical meaning subject to experiment, empirically sufficient, a theory that was able to explain the things we were seeing, and make predictions that were regularly confirmed. Niels Bohr gave a set of lectures in 1922, and Heisenberg, Werner Heisenberg attends one of these lectures, at least one of them, and the story goes on, and I'm not exactly sure how much of the story is apocryphal or how much actually happened, but the story says that Heisenberg at some point objected to this lecture. Now Heisenberg was 20 years old, maybe 21, and Niels Bohr was a Nobel Prize-winning physicist, widely considered to be the whisperer of quantum, someone who intuitively understood quantum mechanics in a way that nobody else did. And here was Heisenberg challenging Bohr. And I think it was expected that people would think that Bohr would be upset about this. But the story says that after the lecture, Heisenberg went on a long walk with Bohr, for many hours, and this had a profound impact on how Heisenberg thought about nature. The story, the legend goes, is that Bohr revealed to Heisenberg that, in Bohr's words, he wasn't convinced that there were particles or orbits at all. You start seeing comments from people like Sommerfeld, and Arnold Sommerfeld was actually Heisenberg's PhD advisor in Munich where Heisenberg was. And then people like Wolfgang Pauli, many of the people who were founding this developing theory were becoming increasingly skeptical that this world picture could survive.

Then in 1925, Heisenberg, who was visiting the leading institution in mathematics and theoretical physics at the time, Göttingen, where there were people like David Hilbert, and there was Max Born, and Felix Klein, leading mathematicians and physicists. Heisenberg was visiting there, and he was visiting Max Born's group, Max Born and his student Pascual Jordan. And he was thinking about all these questions and was terribly perplexed by all of them. There were all these new, more complicated formulas that people were writing down that related energy levels and transitions and atoms to spectra. And he got into this terrible case of hay fever. In the spring of 1925, he went to Heligoland, which is this island where the pollen levels are much lower. And he comes back with this completely strange draft of a paper. And he gives this paper to Max Born and Jordan, and then goes on vacation again. He goes on a bit of a vacation. He's exhausted from all his intellectual efforts. And this paper begins with these remarkably striking statements, statements that a philosopher of science would immediately recognize as pointing to a kind of paradigm shift. Heisenberg begins by saying, we should not think about orbits or particles anymore. We should completely reformulate our physics, our theories, in terms of quantities that are in principle measurable. And so he formulates this different way of thinking about quantum mechanics in terms of abstract objects that Max Born identified as matrices. Maybe people listening to this have heard of a matrix. And the subject to which they belong, which is linear algebra, wasn’t a fundamental part of the physics curriculum at the time. In fact, Heisenberg independently discovered matrices in this work. And their arithmetic operations. And Born realized what Heisenberg was doing. And then together they wrote two papers where they presented this thing called matrix mechanics. In matrix mechanics, there is no picture anymore. There is no picture of atoms and electrons and particles orbiting atoms. There's just this abstract mathematical machinery for predicting the energy levels of things.

Schrödinger comes along a few months later, and through a series of theoretical arguments that I won’t be able to do justice to here that relate to what's called Hamilton-Jacobi theory, which is a beautiful area of classical physics. Heisenberg proposes a different way of thinking about quantum mechanics. He introduces his wave function. And as it appears in these papers. Schrödinger introduces his wave function. This was in early 1926. And in some papers in German and in English he wrote in both languages. He called this theory the wave theory of quantum mechanics, which is a beautiful word, isn’t it? The wave theory, it's a wonderful, a beautiful term. And he introduces his wave function, and as he works with it, he treats it as a mechanical object. He immediately recognizes that it lives in configuration space. He says this in his early papers, but he says maybe this is reality. Maybe reality is this giant undulating mechanical wave in configuration space. And through some mechanism that we don’t quite understand, this undulating wave in this abstract high-dimensional space somehow projects out the facts into three-dimensional reality, facts about where the electrons are. He doesn’t have the complete picture here, but this is what he’s essentially describing. And he’s able to use this wave function as an indirect way to predict energy levels. Max Born comes along very shortly thereafter and suggests that wave functions aren’t mechanical objects. They're mathematical tools. The reason that they’re so intricately valued is that they aren’t really physical things. They’re things that we work with using this kind of mathematical operations to generate probabilities. Wave functions, when you take the complex numbers that describe them and perform this operation on them, what comes out is probabilities. The wave function should be understood as a mathematical machinery for generating probabilities.

By 1928, Schrödinger had already begun to back away from his point of view. He gave a lecture in 1928, his fourth lecture on wave mechanics, where he said that he used to think that wave functions were physical things. He even foreshadows, in some ways, the many-worlds interpretation. And he says it’s an object where everything the system can do plays a role in this giant wave function. But he doesn’t think that anymore. He’s accepted what seems to be the new idea. And then within the next couple of years, 1930, Paul Dirac writes a textbook where he summarizes all the quantum mechanics that was known at the time, and he connects Schrödinger’s wave functions with Heisenberg and Born and Jordan’s matrix mechanics. And he realizes that they’re all part of this deeper reality. There is this mathematical construct. It’s a kind of space called a Hilbert space, which is a kind of vector space. But it’s a vector space that contains complex numbers. And when you look at the vector space in one way, you see wave functions. If you look at the vector space in another way, you see matrices. You see what Heisenberg was doing. They’re all parts of this mathematical structure that’s called a Hilbert space. And then two years later, John von Neumann, the great mathematician, writes a book called Mathematical Foundations of Quantum Mechanics, where he formulates this theory in rigorous mathematical terms. And this is what we’ve had ever since.

Today, when people refer to the Dirac-von Neumann axioms or the axioms of quantum mechanics, they mean this recipe for generating predictions about what we’ll see in experiments, empirical predictions. And this framework relies on Hilbert spaces. Roughly, the first axiom says that every quantum system has this abstract space called a Hilbert space, which isn’t a configuration space. It’s a completely different kind. And that the state, to some extent, of the quantum system is represented by certain kinds of objects in this Hilbert space. State vectors, in the simplest case, density operators, somewhat more complicated. The second axiom is that when the system is left to itself, it evolves according to Schrödinger’s equation, generally, unitary evolution. And then there are three other axioms that talk about when a measurement is performed on the system, and what you’re likely to see. These are called the measurement axioms. And one of them concerns the kind of mathematical objects that represent things that we can measure. These are called observables. One concerns how to generate probabilities for what you’ll see when you make measurements. This is called Born’s rule. And the last one is that after the measurement, the state vector for the quantum state collapses. This is the way we present quantum mechanics to students in textbooks now. If you go to my shelf and look at any of the textbooks on quantum mechanics, it will present something like these five axioms. Different people will sort them differently. They may not say there are five. They may say there are four or six or some other number. But I like to divide them into five, don’t you? So we know we have these axioms. We know they’re very good at giving us a prescription for generating empirical predictions. But what do they tell us about what actually happens in nature? Do they say that the seat of reality is actually an abstract Hilbert space, or a very high-dimensional or infinite-dimensional vector space over the complex numbers? And that the universe is some object in that space that evolves according to some rule. There are some reasons to be skeptical about this. The first is that we can reformulate the axioms of quantum theory in a completely different language. So people who work in other different areas of quantum physics, in mathematical physics, sometimes prefer to formulate quantum mechanics in terms of what are called C-star algebras. C-star algebras aren’t things you find on the beach. They aren’t that kind of C-stars. But they’re abstract collections of mathematical entities that codify how systems evolve in quantum mechanics, how to generate measurement predictions in a way that doesn’t start with Hilbert spaces. When you take this alternative point of view, you get a very general mathematical framework from which the Hilbert space picture can emerge, certainly, well, you can almost always pull out a Hilbert space picture in these stories. Sometimes you get a unique Hilbert space. Sometimes you get a proliferation of different Hilbert spaces, and there are all kinds of deep questions about what that means. But when I started learning about the algebraic formulation of quantum mechanics of quantum mechanics, it opened my mind, as I think it does for a lot of people, to the idea that maybe the Dirac-von Neumann axioms, the ones that we hear about in textbooks, which are axiomatically formulated in terms of Hilbert spaces, may not be the end of the story, and that Hilbert spaces may not be the fundamental way to think about the structure of quantum mechanics. Now, I think one can go much further than that, and maybe we’ll talk a little bit about how one can go further. But once you actually start seeing that there are mathematically equivalent ways of formulating quantum mechanics that don’t start with Hilbert spaces, it makes you somewhat skeptical that the Hilbert space picture is fundamental, and that the objects that are in the Hilbert space picture…state vectors that become wave functions when viewed in a certain way, are fundamental objects.

Well, what allows you to say that Hilbert spaces that can come from the C-star approach means that Hilbert spaces are more illusory? Does the C-star algebra allow you to calculate something that the Hilbert picture doesn’t allow, but it allows you to calculate everything that the Hilbert picture does? Does it then encompass it? So that's an excellent question. In the standard approach to quantum mechanics, the first three axioms of measurement state that everything observable that you might want to look at is represented by a kind of matrix, and more technically, a self-adjoint operator in Hilbert space. It’s kind of like an abstract mathematical entity that you can add other such entities to and multiply together. It acts like a number, like the variable x in algebra, but it’s slightly weirder, in that they don’t always commute with each other. They’re kind of strange objects, but we have a set of mathematical tools for taking them and constructing empirical predictions. When you start with the textbook axioms, you start with a Hilbert space, and then these operators are things that come out of the Hilbert space. In the C-star algebra approach, you go in the opposite direction. You start with an abstract specification of the things that are observable, which are represented by these abstract mathematical symbols. And the symbols have all kinds of mathematical properties, rules for how to add them together, multiply them, just as I said, but they didn’t start as things that live in Hilbert spaces. They have their own existence, their own identity. And there’s also a funny way that people who work in this area sometimes talk about it, where they’ll say, are you a Hilbert space conservative? Meaning, do you consider a Hilbert space to be the starting point, or are you an algebraic imperialist? Do you consider the algebraic things to be the fundamental building blocks? When you start with these fundamental algebraic building blocks, they form a special kind of mathematical structure called a C-star algebra. The C stands for closed, which means there are no holes in the algebra. Everything’s filled in all the way. And the star just refers to the complex conjugation operation, but on these objects, roughly. And there are some other properties that it needs to possess, but they’re very primitive properties. And what you can then show is that there’s a way of constructing a Hilbert space from these algebraic objects. And once you’ve done that, they become matrices or operators in Hilbert space. But once you see this construction, it’s called the Gelfand-Naimark-Segal construction. It’s a beautiful theorem. Once you’ve done that, you start to wonder about how fundamental Hilbert spaces are, especially since some C-star algebras don’t give rise to a unique Hilbert space. And now you have this question about exactly which Hilbert space you’re supposed to use. And hence the Hilbert space proliferation. In a way, yes. So for simple systems, if you have one particle or three particles or 10 particles, it turns out that you essentially get a unique Hilbert space. So you can really think of either one as fundamental. But when you think about systems that have an infinite number of moving parts, any system that you would design as if it had an infinite number of particles, which we do in thermodynamics, or in a quantum field where there are variables at every point in all of space. For that kind of system, you don’t necessarily get a unique Hilbert space. And then there’s this question that’s even more fundamental. And this is an ongoing debate now. It’s called the problem of inequivalent unitary representations of C-star algebras. And it’s an ongoing puzzle in the foundations of mathematical quantum mechanics.

Earlier, we were talking about Heisenberg and how he started with observables, and Schrödinger didn’t. And now you’re talking about taking those observables seriously and forming an algebra out of them only called a C-star algebra. But then you also said that you can take representations of those observables, which become matrices. Yes. And Heisenberg was dealing with matrices. Exactly. Right. Yes. Your picture is exactly right. So this isn’t what I’m working on now. But when you start realizing that you can reformulate the rules of quantum theory in a different way, it kind of opens your mind to other possibilities. I want to know what it opened your mind to. Yes. Where is your mind these days about quantum? That’s an excellent question. So let me go back to the Wigner’s friend thought experiment again, because I think that’s a good link to where I end up, and where I think a lot of people end up. So what was this thought experiment about? In a way, I mean it’s a thought experiment that’s known to a lot of people in physics and certainly philosophy of physics. It’s a really good workhorse in quantum foundations. But I think there are still a lot of lessons to be learned from it. And people have expanded this thought experiment in many different directions. There are so-called extended Wigner’s friend scenarios that a lot of people are working on. But I think the original version of it is really striking. And as I said, it was formulated as far back as 1956 in Hugh Everett’s original thesis. But it’s a very simple question. The axioms say that we have a Hilbert space and that we represent the state of the system as some object in this Hilbert space. The questions that we can ask are represented by observables by these symbols. We can generate probabilities. Then after the measurement we collapse. Why do we collapse? What’s the point of the collapse? The collapse is to guarantee the strength of the measurement outcomes. After you make the measurement, you’re guaranteed to get the same result again. And that’s what the collapse accomplishes. It guarantees that once you get a definite result, if you make the same measurement again before you give the system any chance to evolve further, you’re almost 100% guaranteed to get the same result. Now, the problem of course is, well, what is a measurement? What kinds of physical processes count as measurements? One point of view is that we shouldn’t worry about this. One view is that all scientific theories are ultimately utilitarian projects, meaning they exist only to allow us to relate preparations and setups of measurements and predictions to the outcomes of the measurement or measurement results. That’s all a scientific theory is meant to do. To do anything else, to paint pictures of reality is just fluff. It’s like superstition. It’s not. Necessary. It’s philosophy. It’s metaphysics. It’s not important for physics to do that. And the theory does that. The theory says we make a measurement. And here’s what comes out. The theory does what it’s supposed to do. What more could you ask for? The theory says when we make a measurement, we do this thing, we apply this recipe, and we collapse the quantum state of the system, and then we move on. Everything works out fine. That's your problem.

The Wigner’s friend thought experiment considers observers. The first is Wigner, in the outside world, enjoying a nice sunny day, maybe a beautiful autumn day like today. And Wigner’s friend who’s inside a completely sealed box. Now, people in quantum mechanics might wonder, can you really seal anything in a box? Can’t things get out? Well, particles can leak out of certain kinds of containers via tunneling, but if you make the box thick enough, and impenetrable enough, in a very ideal sense, you can keep everything inside the box, at least for the duration of the experiment. Wigner’s friend is inside this box with a quantum system, some system in a superposition of two possibilities. Let’s say it’s a particle that’s spin up and mixed with spin down in some superposition. Wigner’s friend engages in some interaction that we might call a measurement of this superposed quantum system that’s inside this box. Wigner, on the outside of the sealed box, does not make a measurement, and he doesn’t know in any way what happened. Now we have two observers, and now we have a question. Do we activate the collapse postulate or do we not activate the collapse postulate? We have two observers, and now we have to take a stand, and we have to say whether we’re going to do it or not. And there are only a few possibilities. One possibility is yes, we activate it. We activate this measurement axiom, and the quantum state of the system collapses, even though Wigner on the outside hasn’t seen anything, but it just collapses. For everybody, it just collapses. But now we have a question. Wigner on the outside, why does Wigner think it collapsed? Because Wigner’s friend made a measurement? But how do we know that the thing that Wigner’s friend did qualifies as a measurement? Wigner’s friend is just a physical system in the box. If Wigner’s friend is replaced by an electron that interacts in some way with the other particle, we wouldn’t call that a measurement. If Wigner’s friend is replaced by a tardigrade, a water bear, which is one of these microscopic, extremely resilient creatures that are hard to see with the naked eye, did a measurement take place? Now we just have a physics question. At what point do we cross the line to yes, measurement, or no, not a measurement? Now, there’s something called decoherence which we hear a lot about in these kinds of discussions. Oh, when a large system interacts with a quantum system, there’s something called decoherence. Decoherence causes the quantum state of the observed system to change. But decoherence doesn’t collapse anything. And it doesn’t single out one outcome. And you can actually prove that there’s no way to take the no-collapse axioms in quantum theory and get a definite outcome, to get one outcome singled out from all the others. So the question is, what do we do here? In principle, we need to provide a precise definition of the kinds of processes that count as measurements and which ones don’t, and this is called the measurement problem. We don’t have such a rigorous definition, and we haven’t for a hundred years. Maybe someone will come along someday and say, this is a measurement, and this isn’t a measurement, but at this point, we don’t have that statement. And without that, we just have this ambiguity. This is the famous measurement problem. That’s one possibility. The next possibility is to say that there is a definite outcome to the experiment, but somehow it happens without collapse. Or that the collapse is perspective-dependent. Wigner’s friend sees the collapse, but Wigner on the outside doesn’t see it. But if you take the horn of this dilemma, the trilemma, the quadrilemma, and you go down this path, what you’re saying is that Wigner on the outside has a quantum state that he uses to describe the system, and this quantum state is incomplete. And he doesn’t know the definite outcome that happened in the experiment. And if you accept that the wave function is incomplete, then you’re accepting that there are extra or hidden variables outside the wave function, which, again, doesn’t accord with the textbook formulation of quantum theory, at least according to the usual axioms. The third possibility is to replace measurement with something else, some kind of dynamical process whereby things collapse naturally when they’re large enough, and there’s a whole class of approaches to quantum foundations called dynamical stochastic collapse approaches that do that. The wave function or quantum state is this thing, and for single particles it lasts for a very long time before it collapses, but when you get enough particles together, if any one of them collapses, then everything collapses, and that explains why very large objects collapse very quickly. That’s the third option. These approaches are very interesting. Some of them have been experimentally tested. They involve a lot of new parameters, you know, and you have to guess the equations, and we don’t really have a well-motivated way of understanding which one to use. The fourth possibility is that there’s no outcome. Nobody gets

On a specific measurement. Either there is no outcome at all, which doesn’t really seem logical, or all outcomes happen in some sense, and that’s what Hugh Everett argued in his thesis when he looked at this problem. There is no collapse. We get rid of the collapse postulate, and Wigner’s friend splits into two parallel versions, each seeing a different outcome. One sees spin up. One can see spin down, and this is inside the box. Also, Wigner on the outside doesn’t register a definite outcome. If the box is opened, Wigner sees the definite outcome, and Wigner splits into two versions, and the universe sort of unravels with the copies spreading out and branching, or the wave function of the universe splits into multiple possibilities. This is the many-worlds interpretation, and it’s a source of ongoing contention. There are lots of questions about how to understand fundamental questions about identity and the philosophy of mind in a world like that. It’s not clear how to understand probability in a world like that. There are ongoing debates about how to accomplish that.

This is basically what we’re left with. This is what Wigner’s friend experiment left you with. You have to make a decision. I think there’s a fifth possibility, which is that quantum mechanics is wrong. And until we have good reason to believe that it’s just wrong, maybe we should work on the assumption that we need to make the theory work better. I think that’s a good motivation to take seriously the fact that something is wrong with the postulates of quantum mechanics, and that we need to do something about it. It’s not just ad hoc, it’s somehow obscure or incomplete or inconsistent in some way, and we need to fix it. Where does that leave me, and where do I go?

One way to understand what I’m trying to do is to go back to 1922, when people were willing to give up the old pictures of particles and fields and actual things being there, and doing things, and behaving. Just as people were about to give up on finding the laws, and behaving based on those constituents that would give you a sufficiently empirically adequate theory, maybe it’s premature to give up. Maybe we need to think more generally about what form the laws could take. By 1922, people already realized that it’s possible to have laws that are stochastic and probabilistic. Laws that didn’t tell you in a definite and deterministic way what was going to happen, but simply told you probabilistically what was going to happen. We knew about these. Einstein certainly knew about these kinds of models. One of his great papers in 1905 was about Brownian motion. That helped provide strong evidence for the existence of atoms, it even allowed you to know the size of the atoms, and this is a model that relies on what’s called a random process. Stochastic from the Greek word stochastikos, which means aim or guess. These are models where the laws predict probabilistic behavior. People knew about these things. Markov had introduced the Markov matrix in the early 1900s, 1906 or something like that. These ideas were in the air, and people were thinking, well, quantum mechanics behaves in an unpredictable way. Maybe the unpredictability can be absorbed into probabilistic laws, but people couldn’t come up with the right set of probabilistic laws at the time.

Over the years, people have tried to come back to this question. In fact, in Hugh Everett’s thesis, he talks about an effort by a physicist named Fritz Bopp who replaced the usual formalism of quantum mechanics with probabilistic laws, where particles move according to probabilistic laws. Everett actually said this was a promising direction. He didn’t want to study it in his thesis, but this was worth paying attention to. And many other people have worked on these proposals, like Emery Fényes in the 1950s, and most famously Edward Nelson in the 1960s, but the models were extremely complicated. To get the same predictions that you get from quantum mechanics theory, you had to—and this is the only political thing I’ll say—gerrymander the districts. You had to manipulate—I mean this in the same metaphorical sense. You had to manipulate the laws. You had to start with what we knew was supposed to be predicted and then work backward and engineer these incredibly complicated stochastic laws in order to get the same predictions. I felt very unmotivated. The spirit was a very interesting spirit. Let’s suppose particles bouncing around or fields, whatever, evolve without Hilbert space, without wave functions. Let’s just give them directly probabilistic rules and in a way that makes those probabilities agree with the probabilities that we see for the outcomes of measurements in our experiments. But implementing the idea was hard. And it wasn’t clear that we had the right laws to do it. And the laws that make it work even in the simplest cases require a lot of fine-tuning and manipulation. And when you look, for example, at Nelson’s stochastic mechanics, his laws are extremely complicated. And you actually have to find the wave function first from the Schrödinger equation, and then plug it into these laws, and the laws involve rates that move back and forth—it’s incredibly complicated. And there were competing approaches. So, Louis de Broglie introduced the pilot wave theory in the 1920s which David Bohm independently discovered. And these theories were originally built on hidden variables, particles, that were somehow guided by the wave function. When you try to extend these models to anything beyond nonrelativistic particle systems but, like relativistic systems and fields, it gets extremely complicated and you end up deducing probabilities and stochastic laws again. So it seems that there’s an understanding that ultimately, to describe most kinds of quantum systems, we’re going to need something like laws that are probabilistic in nature. But none of the probabilistic laws that people know seem to work very well. Either they didn’t work or they needed so much fine-tuning that they seemed unmotivated.

In 2022, I was trying to teach a class, and I got a problem. We got to a point in the class where I was trying to present a little bit of quantum mechanics, and I was trying to decide how to do it. And I’ve taught this class many times. Which class? It’s called Introduction to Theoretical Physics. It’s Physics 19 here at Harvard University. And we cover a lot of topics in the foundations of theoretical physics in this class. During the semester, we present some concepts from quantum mechanics, but this is almost at the end of the semester, and I’m trying to give them a nice, self-contained preview of quantum mechanics. But these students are mostly very new to physics. Many of them have just started learning college-level mathematics. I can’t start talking about Hilbert spaces or C-star algebras or GNS theory or any of that stuff. I need to find a more gentle way to get into quantum mechanics. And every year, I would rip up my syllabus and try again. I was never satisfied with the way I was doing it. Sometimes I would say, maybe we’ll start with wave functions. Sometimes I would say, maybe we should start with matrices. I couldn’t come up with the way I wanted to do it. And in the fall of 2022, I remembered some projects that I’d worked on when I was an undergraduate and even in graduate school where I needed to use the theory of stochastic processes for, you know. As an undergraduate, it was a special extra homework assignment for a linear algebra class. In graduate school, I actually used it in simulations. And, you know, the theory of Markov chains and stochastic processes, it bears some very vague similarities to quantum mechanics. It has probabilistic laws. You represent the probabilities of the system using a vector. The time evolution is represented by matrices. Now, they’re all a little bit different. The probabilities don’t involve complex numbers. The matrices are what are called stochastic matrices. But there are these similarities. And so I sat down and said to myself, maybe I can find a way to put these two theories together. The theory of stochastic processes on the one hand and the theory of quantum mechanics on the other. I had this kind of picture in my mind as if I’m walking on the edge of a cliff. On one side, you had a kind of classical world, classical stochastic processes, probabilistic processes. And on the other side, you had quantum mechanics. And there was this huge gap between them. And I was looking for a place where the cliffs get closer together and where it’s easier to jump from one to the other. So I started modifying the mathematics of stochastic processes theory and quantum mechanics to try to bring these two pictures closer and closer together. I was doing this without knowing much about this previous work that had been done. The work of Bopp and Fényes and Nelson. If I had read all those papers, I would have given up because I would have decided it was impossible. But sometimes you can only do something if you don’t know that it’s impossible. I didn’t know it was impossible. So I brought these theories closer and closer together expecting that there would be a moment where I would need to do some kind of jump to go from one theory, classical physics, to the other theory, quantum physics. And the strangest thing happened. The faces of the cliff merged, and there was no longer a chasm. For those who are familiar with second-order phase transitions in statistical mechanics, it was as if I reached a second-order phase transition point. On the one hand, you have the two phases for some materials like water, right? And then you cross a point and the phases merge and there’s only one phase, right? There’s no longer a difference between liquid and gas anymore, right? And I didn’t understand what happened. I was puzzled. So I started looking at the literature to see if there were other people who had done what I did and nobody had done what I did. I think nobody thought it was worthwhile to try to do this thing that I was doing. But I started reading some of the literature on stochastic processes and Nelson’s work and I was puzzled. Why hadn’t they succeeded when this seemed to succeed? And that was around October of 2022. I know the reason. I went to the office of one of my colleagues, the great Logan McCarty, who runs science education here at Harvard. He’s a quantum chemist. He knows quantum mechanics backward and forward. And I went and talked to him and I went to him, and he had a whiteboard and I started writing down everything that I was doing. Because at that point I realized that the secret sauce, the thing that made this work is that I implicitly gave up an assumption that I didn’t even know was supposed to be made or that anybody had made. When people model stochastic processes, they usually assume that the process is a Markov process, named after Markov, like Markov, the Russian mathematician who introduced Markov matrices. A Markov process is a process if you want to know what’s going to happen to your system, all you need to know is what’s happening right now. And then the laws tell you, at least probabilistically, what’s going to happen next. It’s memoryless. Memoryless, exactly. It’s memoryless. Now, Brownian motion is memoryless. That was Einstein’s random model of atoms. And generalizations of Brownian motion, and Wiener processes, and many of the familiar processes that we’ve heard about, random walks, Poisson processes, these are all Markov processes. Nelson’s work, Markov processes, they’re all memoryless processes. Now, I was vaguely aware that there was a subject called non-Markov stochastic processes. And I went and looked in textbooks to see if anybody had done what I was doing. And usually at the end of the textbook, the book says something like this is all there is to stochastic processes theory. There’s a bigger subject called non-Markov stochastic processes theory, but this is extremely complicated. And it’s beyond the scope of this book. So it was very hard to find any books on non-Markov processes. And I searched the literature and discovered that some people were trying to develop tools to deal with non-Markov processes. There are some that can be transformed into Markov processes with a change of variables. They become what are called hidden Markov processes. And some, if they’re sufficiently non-Markovian, they’re very hard to deal with. And so it seems like the wild west. There’s just this regime beyond the Markov approximation that not many people have really felt comfortable exploring. It wasn’t clear how much we needed them. I mean Markov chains are really good at modeling the world around us. Do we really need to spend a lot of time worrying about non-Markov processes? And what I realized is that I had inadvertently introduced a kind of non-Markov process without realizing it. And as I dug into this more, I discovered that what I was working with wasn’t even a non-Markov process in the most common sense. It was a process of a more general kind that first appeared in the research literature in 2006. And it’s called an indecomposable process. Indecomposable processes are in some ways the most general kinds of rules, dynamical laws, probabilistic generally, that fail to possess the Markov property. They’re more general than what we usually consider to be non-Markov processes. I think there’s more than one way to be non-Markovian, and people had a very particular idea of what they meant by non-Markovian. And this is more like a non-Markovian process than a non-Markovian process. These processes first appeared when people were trying to understand the kinds of dynamical rules that describe Hilbert space-like objects, so not what was on my mind. They were actually in Hilbert space trying to describe how things in Hilbert space could evolve over time. And for them, the indecomposable process was just a law that could tell you how your process would evolve from one time to another, but it cannot be decomposed into smaller times. So you have a law that tells you how to go from zero to final time, but the theory doesn’t give you laws for intermediate times. That’s it. That’s indecomposability. Certainly not laws that can be iterated and composed into a larger law. And if you think about it, this is certainly more general than divisible laws. I mean, because there’s a larger class where we don’t have some properties that we might want to have, which is divisibility. Newtonian mechanics is divisible. You evolve a system from an initial time to an intermediate time, and then you can stop, look at the state of the system now, and then, if you want, you can use Newton’s laws to tell you what’s going to happen next. Nice and divisible. Maxwell’s equations that describe the evolution of electromagnetic fields are beautifully divisible. And quantum mechanics, as it’s usually formulated in terms of the Schrödinger equation, is beautifully divisible. The Schrödinger equation tells you how the quantum state will evolve, and you can evolve it as far as you want, and then read off the quantum state somehow, and then predict where it’s going to go next. Is there another way to say indecomposability, or sorry, divisibility as a continuous time parameter? That’s a good question. Not necessarily. Because you can imagine an indecomposable process where you are allowed to go from the initial time to any time you want, so time is continuous. But once you pick one of those times, you won’t have a law that tells you how to go from that time to later times. So time is not necessarily discrete here. That’s an excellent question. But what you lack is the ability to divide time into smaller time intervals. Yes. So we don’t have a lot of experience with this kind of law until this research paper came out, titled Decomposing Quantum Channels by Ignacio Cirac and Michael Wolf. Ultimately, it was published in 2008 in the journal Communications in Mathematical Physics, and people are playing with it. You know, the paper gets a lot of citations. It wasn’t until 2021 until some people put forward the idea of a classical process that could be indecomposable, like just a process where the system has some configuration, and it changes, and it behaves probabilistically, but the way it behaves probabilistically is indecomposable. This shows up in a figure on page 15 of this beautiful review article written by Mills and Modi, where they mention the idea that such a process is possible. I didn’t know it existed. I came up with the term indecomposable independently because I didn’t want to use irreducible, which is used in many ways in physics. I didn’t want to use it, I was just looking for a word that would give you, and evoke the right idea. And it turns out that they use the exact same terminology. So this was a year before I did it. And they didn’t connect it to quantum mechanics. They were just mentioning, oh, it’s possible, theoretically, to have a classical process with probabilistic behavior, where the laws aren’t rich enough. They’re very sparse. They won’t tell you how to go from any time to any other time. You can only evolve from certain times to other times. The process is overall indecomposable. What I inadvertently did is discover these indecomposable stochastic processes and showed that there’s just a change of mathematical representation that would take such a system with classically looking configurations, and it can be a system of particles, it can be a system of fields, whatever we do ‘re trying to model. And if it has indecomposable stochastic laws, then through this mathematical representation, this correspondence, you get Hilbert space.

That’s very interesting. Yes. And all the strangeness of Hilbert space, superposition of states, and interference effects, these are all tied to what’s called coherence, phases, right? All these things are tied to superposition and interference and coherence. These things can now be understood. They can be given meaning. So, in quantum theory, these things don’t have meanings. They’re just pieces of mathematics that appear in the formalism. We don’t understand what they mean. We know that interference shows up in experiments. We know that these things are called phases or coherence or superpositions, and we see these showing up everywhere. But they don’t have a direct physical meaning. We don’t know what they represent or what they really mean. What I was able to show is that by going from this concrete physical picture of things with clear existence, probabilistic evolution in this indecomposable way, by going from that picture to a picture with the beautiful Schrödinger equation, where the evolution is divisible, it’s a nice differential equation, and very familiar mathematics, when you go to that seemingly divisible picture, you have to give something up. The indecomposability doesn’t disappear. It becomes all those strange phases and interference effects and coherence. That’s how they show up in this picture. It’s as if there’s no free lunch. If you want a mathematically convenient and simple picture, a picture where you can evolve the system for any amount of time you want, pause, and re-evolve, a nice divisible dynamical picture, you have to pay a price. And the price is that all this indecomposability becomes these strange phases. In a very loose sense, if you want to think roughly of these indecomposable processes as being like processes with memory, all those interferences and phases, that’s the memory encoded in this strange way. And that’s the picture. That’s beautiful. Yes. So in this picture, there’s no fundamental rule for the wave function. I mean you have things bouncing around according to laws that are more general than Newtonian mechanics laws or even Markov process laws. And if you want to design a model of measurement, you just design another system that interacts with your first system, and give the whole thing a nice simple set of indecomposable stochastic dynamics. And what you will find is that the measuring apparatus will evolve randomly to one of its final readout configurations with exactly the right probability that we expect from Born’s rule. But there’s no wave function. There’s no collapse. It’s just one big random process happening.

So what influences the choice of the time interval to define that time? You said that you can choose any time interval, but then you can’t choose a subinterval. Right. That’s an excellent question. So, in order to start an indecomposable stochastic process, you need a time that you can start at, right? And then the process probabilistically tells you where the system is going to end up at any smooth later choice of time. But is there only one special time that you can start things? The answer is no. What you can show is that when you have a system that interacts with a larger system, such that it’s not left alone, it interacts with a larger system, what you find is that if this larger system exchanges information, with your original system, as would happen if you have a measuring apparatus that reads the configuration of your system, when it does, you just let the stochastic process tell you what’s happening, right? There’s no postulate here. I just let it evolve. And what you discover is that you get a division event. The evolution splits at that moment, from the point of view of the smaller system. And now there’s a new time where the evolution is discontinuous, and you can start again now from that new time. So, in the indecomposable stochastic process, it doesn’t mean that there aren’t times where you can divide. There are times where you can decompose the evolution, but the times aren’t given to you at the beginning. They’re not fundamental. They arise from the mutual interactions between systems. Now, when you ask how this looks from the point of view of Hilbert space, in the standard picture of quantum mechanics, from the point of view of Hilbert space, how do those division events look? They look exactly like decoherence. So decoherence, again, is when a large system interacts with a smaller system, the environment comes, interacts with the system, exchanges information, and we get these changes in the quantum state of the other system. Changes that don’t pin down a single answer, but somehow change it. These changes wipe out the coherence effects. They kill the interference effects. From the point of view of the indecomposable stochastic process, it looks like a generation of a new moment, a new division event, an event where you can start the evolution again. And this happens at the end of measurements, and this is a physical process in this picture, and it coincides with what we might call measurement. But it’s generated by the laws from this picture themselves. Okay, I’m puzzled. So division events, are they just another renaming of what measurement is? Good question. So, in a way, if you give me a second system that I design explicitly. So let me phrase it differently. In the textbook postulates of quantum theory, measuring apparatuses are treated as being outside of the formalism. They’re sort of in the background. They’re sort of like the person behind the curtain, right? We do a measurement, and we just change the quantum state of the system that’s being measured, but we don’t usually put the measuring apparatus explicitly into the description itself. If you try to do that, if you try to put the measuring apparatus into the description, what you find after the measurement is that the measuring apparatus and its possible readout configurations become entangled with the possible configurations of the system that’s being measured. There’s this entanglement that happens. And the question is, well, then what? Now they’re entangled, what do I do? What you’re supposed to do is collapse to one of those entangled branches. That’s the collapse. Now, if there’s a larger environment that comes along, and the larger environment looks at the measuring apparatus, it becomes entangled with the measuring apparatus. And there’s this process, this kind of formal process where you can sort of ignore the environment. And you see that there’s a suppression of all the interference effects, but there’s not yet a determination of a single outcome. You have to at some point pull out the fifth postulate and activate it and collapse to one thing at some point. In this indecomposable picture, you can bring in this measuring apparatus, you can bring it in, and you let the whole system evolve stochastically. And what you find is that, by introducing it into the environment, the environment comes along, and you can treat them all as actual systems. None of them are outside the formalism, they’re all designed as systems in the formalism. And what you find is that when you let these systems evolve, the system being measured and the measuring apparatus evolve probabilistically toward the configurations that we expect from the collapse postulate. And because of the environment, you can show that the measuring apparatuses in the objective system, they undergo a division event at the same moment of interaction that we call measurement. And this division event means that you can now restart the laws and start applying them from this moment. This division event isn’t assumed, it’s not a postulate. It’s something that just emerges from the usual rules of probability theory, when you actually model the systems explicitly. So it is measurement in a way, but it doesn’t require a conscious being, it doesn’t require a human, and any system that has a very large number of moving parts, we call them degrees of freedom, in contact with a nice large environment with a large number of degrees of freedom, will work at a very high level of accuracy, and it will produce measurement outcomes that probabilistically match what we expect from textbook quantum theory, and it will generate a moment where the stochastic dynamics of the universe. The system being measured in the measuring apparatus can be restarted and have its own dynamics. So yes, it’s measurement in a way, but it allows us to go inside measurement. It allows us to see measurement actually happening, and it gives us a way to understand why it’s happening.

Measurements with this thing, instead of just assuming that they lead to the collapse of wave functions. Do you need to mention three different systems? Then the environment, the apparatus, and the measured system? Yes, that's right. And we require that in standard quantum measurement theory anyway, in the usual textbook theory. And the environment as well? The environment is required, yes. Yes. Because the usual argument is that it is the environment that generates the decoherence between the measuring apparatus and the system being measured. But when you say environment, I mean that in any realistic measuring apparatus, you have, you know, a part of the measuring apparatus that actually participates directly in the measurement, but then you have the rest of the measuring apparatus. I mean, it's a large body, it's in a box, it has dials, it has other experimental things. I'm sorry, this means I won't be a good experimentalist again. But this, yes, there's almost always an environment.

So, you first have to describe the environment, the apparatus, and the measured system classically, and then you do some version of the quantum measurement procedure? Yes, so you can design them classically, meaning that they have classically-like configurations. The measuring apparatus is some system, or physical body that can be in different possible configurations. Among those that we list at some level of coarse-graining, we list, you know, the system is empty, and it doesn't show a reading yet, and the system shows an up spin on the dial, and the system shows a down spin on the dial, you know, but these are concrete configurations made out of certain configurations of constituent atoms, whatever you want. And that means their configurations, their ontology, their physical existence is conceived of in a pretty classical way. The quantum part is that the laws that they obey are not deterministic differential equations like Newton's laws, or Maxwell's laws of electromagnetism. And the laws that we give them are these non-separable stochastic laws, which we don't usually see in classical systems. The claim is that when you give these systems these more general kinds of laws, that's enough to make them behave like quantum systems.

So what is stochastic or random here? Good question. So all these systems have configurations, and these configurations evolve according to stochastic laws. That means that the configurations change over time in a probabilistic way. And now, why do they change in a probabilistic way? I don't know. These are axioms, right? At some point, you have a physical theory, you have to propose some axioms. I mean there's a position in how we think about fundamental questions in philosophy called foundationalism. You have to start somewhere. The standard textbook version of quantum theory, which works extremely well, and I certainly don't claim that any of the predictions it makes are wrong. They're all spot on. These axioms are axioms, and they are quite strange. We assume Hilbert spaces and state vectors or density matrices and the Schrödinger equation and commuting operators and all the collapse postulates, and we assume them because using these axioms we get the empirically correct predictions. This picture that I present has axioms as well. The axioms are simpler. Each system has a set of possible configurations, classical-like configurations, and the laws are these non-separable stochastic laws. That's it. Now, if you want to ask why the laws are non-separable stochastic, I don't know. The best I can say is that when we try to explain things, there's an argument that you want to start with the least restrictive probabilities, the most general probabilities, and then argue that under certain circumstances you get more restricted probabilities. Non-separable stochastic laws are the most general kinds of laws that you can imagine. Maybe you can go beyond that. I don't want to say that they're definitely the most general, but they're certainly more general than Markovian laws. They're more general than the usual way we think about non-Markovian laws. They're more general than deterministic laws, laws of the differential equation type. To me, it makes sense that you would start with the most general kind of law, and then explain why we expect under certain special circumstances to see Markovian behavior, and why we expect under certain circumstances to see deterministic second-order differential equation type of law. That's the task of understanding the classical-quantum transition, understanding the so-called classical limit. How do we go from the laws of quantum mechanics to laws that look like the laws of Newtonian mechanics or look like classical Markovian processes? These arguments also continue in this picture. We don't start with deterministic laws. We don't start with Markov chains or anything like that. We start with these more general laws, and then we adapt the usual arguments that people usually use in trying to do this to get to the classical limit, the classical regime, we see why under certain circumstances we get large systems that have a classical appearance whose laws appear to be deterministic or at least appear to be probabilistic Markovian laws. To me, this is the right direction for interpretation. You start with the most general, least restrictive kinds of laws, and then you go to more restrictive laws.

I guess I'll say that I don't feel like I need to explain why particles or things behave probabilistically. Probabilistic behavior is the default behavior. That's what happens if we don't have a good reason to believe that the laws should be deterministic. And then the goal is to clarify why under certain circumstances we get deterministic predictable behavior. But the truth is that I don't have a deeper explanation for why we expect particles to wiggle probabilistically. That's what it means to have a non-separable stochastic law or a stochastic law in general. And your probabilities, are they classical probabilities or are they quantum amplitudes? That's a great question. They're classical probabilities, and that's kind of key. So this basic picture, the claim is that every quantum system at its beating heart is really a system with configurations. If it's a particle, then we're talking about where the particle can be. If it's a field, then we're talking about the different intensities that the field can have in space. If it's a system of abstract bits, like in a computer, it's a configuration of zeros and ones. That's what I mean when I say a classical-like ontology or a classical-like configuration space. And probabilistic laws specify ordinary probabilities for these things. They say the system starts in some configuration, and then the non-separable stochastic law takes the form of laws that tell you the likely place where it'll end up. And these are just ordinary probabilities. They're real numbers between zero and one. They sum to one. There are no complex numbers at all in this picture at this stage. But if you want to describe the same process in a mathematically more convenient way, I mean, non-separable processes are actually somewhat inconvenient to work with, especially if your system is complicated, because we don't have this ability to factorize the laws. It's very hard to use them directly. So instead, we use this mathematical change of representation, the Hilbert space picture. We translate all the components in this basic process into the Hilbert space picture. In the Hilbert space picture, we have nice differential equation type laws, the Schrödinger equation. We have complex numbers appearing. And then the probabilities that you have in the non-separable stochastic theory become what we call the squared magnitudes of complex numbers. The complex numbers whose squared magnitude you take to get the probabilities are called amplitudes. So only when you go to the Hilbert space picture do you need amplitudes. And amplitudes do all kinds of funny things. They interfere with each other. If you want to write down a process where the system could go one way or the other, you're supposed to consider all the possibilities and then sum all the amplitudes together, and there's interference, right? But all of this happens in this kind of mathematical representation, this Hilbert space side of things. And the claim is that this doesn't tell you what's actually happening in the system. This is just extremely convenient mathematics. And again, it gives all the right predictions, right? The idea here is that you recover the standard axioms of quantum theory, but you now understand where they came from. And now they're no longer inconsistent anymore, because at the fundamental level, there's no special role for the observer. There's no special role for measurements. All the processes that we think about in quantum mechanics are just ordinary processes. They appear strange and bizarre when you see them on the Hilbert space side of things.

Have you encountered any resistance from people who like the mysterious quantum mechanics? Yes. Explain. Well, I mean, quantum mechanics is very exciting. Actually, let me rephrase that. It doesn't mean that they like their quantum mechanics mystical. Have you encountered any resistance from people who like the mysterious quantum mechanics? Puzzling. Yes, they like it to be shaken not stirred, right? I mean a little bit. Yes, I think there's a sense that this picture is deflationary. It's deflationary. It takes away the bizarre statements. Oh, the cat is alive and dead. And it deflates them and says, actually, it's really one or the other. And we just represent it mathematically with a state that looks like it's a superposition of alive and dead because it gives us a factorizable evolution that we can use. It's easier to study mathematically, but that's not reality. It deflates that. Yes. Just to clarify, deflationary is a philosopher's term for saying there's nothing to see here, folks. Just calm down. Yes, it's basically like sucking the air out of the balloon, right? It brings it down to earth, and that's basically what this does. I think it makes quantum mechanics, in some ways, more boring. And you know, as I was working on this, I kind of felt that when young people come into physics and they start learning about quantum mechanics, it's extremely exciting to think that you might be able to understand what's happening when Schrödinger's cat is alive and dead. And you know, particles can go through both holes in the double-slit experiment simultaneously and interfere with themselves. And you know, everything is entangled with everything else. And maybe when we think about things, we humans are special. We collapse things. That's boring quantum mechanics. It says that at some level, quantum systems, if things go right, might just be kind of mundane. They're just there's no Hilbert space wave function. There are just things, atoms, chairs, whatever. And they just evolve in a non-deterministic way, in a very non-deterministic way. Okay, still probabilistic. We can still assign probabilities to things because you can imagine laws that are so non-deterministic that we can't even assign probabilities to them. But we can assign probabilities to them. But then a lot of the magic has gone away, right? There's no special role for humans. In a sense. Meaning what? The history of science has been pushing humans further and further away from the center of the story. You know, initially we were the center of the universe. Then we weren't. Then maybe we were the center of all observation and quantum mechanics. And this takes us out of that completely. There's nothing particularly special about human observation or the human mind. And I think as we evolve, we may not soon become the smartest people on this planet, the smartest beings or entities on this planet. Um, so I don't know. I mean I can only speak for myself. I felt a little bit sad when I thought that there is a way to think about quantum mechanics where there is no wave function. Nothing branches into multiple realities.

So you mean there's no fundamental emergent wave function? Well, okay. It emerged. Yes. So wave functions play their role, so there's a way to take Newtonian physics and introduce kinetic energy, which is energy associated with things moving, and potential energy, which is energy associated with the configurations of things. And you can take these two quantities. They're more abstract than being like particles and forces and things, but you can imagine kinetic energy and potential energy. And you subtract them from each other, which is kind of weird. Usually we think total energy is the sum of these things. You subtract them and then you can integrate this thing over time. And you get a mathematical construct called the action, the action functional. We actually cover this in my class now. The thing that you integrate kinetic energy minus potential energy is called the Lagrangian. At least in the simplest cases, it's kinetic minus potential energy. And you can use this to encode the laws of your system. You can use this to describe how the system behaves. But you can ask what is the action? Like, is it a physical object? You can make the action evolve in time. You get what's called the principal Hamilton function. It obeys a partial differential equation. And I didn't mention this, but this is the beginning of Hamilton-Jacobi theory. And by studying the action evolving in time, and the principal Hamilton function, and its partial differential equation, Schrödinger was led to discover the Schrödinger equation. It was a beautiful extrapolation of the partial differential equation obeyed by this action. So this action is a function of time or principal, the principal Hamilton function is very much like the wave function in quantum mechanics. So much so, as I said, that it inspired Schrödinger to develop the wave function. In a sense, Schrödinger's wave function was an exponential of the principal Hamilton function. And now you can wonder, what is the meaning of the principal Hamilton function? Is there a physical body in the world that evolves according to this partial differential equation, the Hamilton-Jacobi equation, the equation satisfied by this principal function, this integrated kinetic minus potential energy? I think most physicists would say no. They would say it's just a mathematical convenience. It's an auxiliary to the theory. We can use it to generate predictions. We can use it to predict. And that's great. It's beautiful mathematics. But there's nothing in physical reality that it describes. And what I'm saying is that the wave function is basically like that. It belongs to the same conceptual or metaphysical category as Hamilton's principle does. Sure, it's in the mathematics. It's convenient. But there's nothing physical that it describes. And because there's nothing physical, there's nothing that branches. There's nothing that splits into branches. So you would never think about the possibility of multiple worlds or anything like that. There's nothing in the ontology of the picture as I presented it that would play the role of branching realities. So things like many worlds don't even get off the ground in this picture. So I think that's what I mean when I say there's no wave function. I mean it in this somewhat precise sense. I don't mean that we shouldn't use wave functions in the mathematics or use them for prediction. But we should be a little more humble in the way we talk about them.

Have you talked to Sean Carroll about this? That's an interesting question. Not yet. But at some point we will hopefully talk. Yes. It's an interesting question because I haven't traveled much in the last year or so. So our paths haven't really crossed. But I would like to have a conversation about this at some point. Any Everettians or many-worlds? People? A little bit. A little bit. But I think you talked about the mysticism. You talked about how people like their physics. The fact is that people get good at doing physics in a certain way. And we develop intuitions. When you learn to play a musical instrument, you learn to play the piano, you learn to play chess, you learn gourmet cooking. And you do this over many years. And if you have the right mix of motivation and talent and skill and the right environment and the right mentors, the right mix of all these factors, you will end up becoming very good at it. And you can do wonderful things. You can play beautiful pieces of music. You can prepare a wonderful kitchen. You can become a grandmaster in chess. And if somebody comes along and says, I think you should do it differently. You should go back to the starting board. You should sit at the piano bench like this. And you should hold your hands like this. Or you should play chess. And you should start the pieces in a different position. And we should change all the rules. The chess pieces should move differently. You're going to put the person back to square one. And you can imagine why somebody might not want to do that. And not for any negative reason. It's just that they've honed themselves to do really good work. And asking them to go back to the drawing board and start all over again is a big ask. And so I would basically say that anybody who's been working in quantum mechanics in a certain way for their whole career, who's developed all kinds of intuitions, have a good feeling for what to do with wave functions and things. You know, I can imagine somebody, in that position, being hesitant to go back and rethink all the foundations from scratch. Especially if what they've heard for most of their career is that the foundations are fine. Quantum mechanics is just a tool for generating predictions. Why should I go back and think about any of these things? So I don't think I'm still doing that, I'm not criticizing any of that position. But what I've definitely found is that newer people in physics, students, tend to be more interested in talking about a project like this. But there are absolute exceptions among professors as well. I mean, I have, you know, some wonderful colleagues who have been incredibly receptive and open, and we've talked about this project a lot. And I've gotten really good input from people. So, you know, but I think it's a mix of intuition-building and also what draws us to study quantum mechanics. I mean quantum mechanics is a weird theory. I mean, the belief that you're going to spend your time imagining the world is a wave function or a state vector or some abstract object living in a high-dimensional configuration space or in the somewhat modern sense in a high-dimensional Hilbert space. Hilbert spaces are beautiful. The mathematics is incredibly elegant and has a certain kind of spice to it. But you get complex numbers floating around. We don't know why, you know, at least in the way that we usually talk about it. I can imagine being romantic about that. I was. I feel that's what drew me to quantum mechanics. So I can certainly see why some people would want to hold on to that. And I think among all the people who take a romantic view of quantum mechanics, I don't think anybody takes a more romantic view than the Everettians, the people who work on the many-worlds interpretation because their picture of the world is not only weird. It's deep, right? The idea that every time a quantum event happens, the universe splits into two branches or three branches, but in a more sophisticated and more precise version of Everett's approach, it's an endless profusion of branches. I mean that's stunning and amazing and forces you to confront some very deep questions in metaphysics, in the philosophy of mind. It's just very rich with inquiry. I can imagine people who work in this area wanting to hold on to it.

So I'll give you two examples because I can relate to this, being stubborn and pig-headed. So when I was learning about the categorization of quantum mechanics and reading papers like, okay, here's another way you can think about quantum theory as a dagger-compact monoidal category. And I look at it and think, okay, but so what? I still have that, so what? But I think part of it is that I'm used to doing quantum mechanics in an alternative way. Now, my dad sets up his office in such a way that if you move one piece of paper, he'll get upset at you, even if it's out of the way, he keeps everything in his mind. I had to install Windows XP two years ago, Windows XP on his computer because he's familiar with Windows XP and wants to use DOS and can't run QBasic on Windows 10 without some emulator or virtual machine. So my dad is kind of like, what you would categorize as, or what you would describe as some people who are used to quantum mechanics the old way, at least when it comes to not just disliking it for being deflationary, but disliking it for being different. Now the question might be, okay Jacob, what does this give me? What does your approach give? Why should I learn it? You just said that it looks like ordinary quantum mechanics. Okay, great. Maybe to you it's a more beautiful version, and maybe to someone else more complicated, but whatever, what does it give you? What's the payoff? That's a great question. So the first payoff is a rectification of the axioms. The axioms had this mystery or inconsistency or incompleteness to them. They had this puzzle about what measurement is that seems to play a role in the axioms, but isn't defined by the axioms. And you get problematic thought experiments like the Wigner's friend experiment. And the Wigner's friend thought experiment is that it's not the same as the black hole information loss problem by any means, but there are some similarities. There's this big problem in quantum gravity. Black holes seem to radiate according to work going back to Stephen Hawking. And there's this tension between black hole evaporation quantum mechanics and unitary evolution or Schrödinger evolution, which is information preserving in some sense in quantum mechanics. And the fact that black holes are supposedly not allowed to let information escape. There's this tension between all these things. That's the kind of tension we see in thought experiments like the Wigner's friend thought experiment. And it's not as if the black hole evaporation problem is easily accessible experimentally. I mean, to go out and do experiments on black hole evaporation, you would have to prepare 10,000 similarly prepared black holes, which we don't have to do. Wait several times the age of the universe for all of them to evaporate, and make sure that they all evaporate inside a perfectly closed room with perfect detection mechanisms to collect all of the Hawking radiation coming out with sufficient sensitivity that we can detect, you know, incredibly tiny entanglement effects, right? And then we subject all the data to a level of analysis that we think might be computationally impractical. I mean, this is not a practical problem, at least as far as we know so far. That could change. But as far as we know, it's not a practical problem. Yet, this is considered a serious area of research. Whereas inconsistencies in quantum mechanics are not considered serious for some reason. I don't quite understand. There's some sociology there. But what I would say to quantum physicists is that one of the payoffs of this approach is that it resolves some of those inconsistencies, which is maybe in itself a good thing. It does so in a way that doesn't require that we only model one kind of quantum system. One of the drawbacks of pilot-wave theory like Bohmian theory is that it works really well with very simple kinds of systems that consist of a fixed finite number of non-relativistic particles, but it has resisted precise generalization beyond that. So it's nice to have a more standalone and simpler model than the Bohmian approach. And it doesn't face many of the confusing metaphysical puzzles that one has to deal with with Everettian quantum mechanics. You know, we're not making millions of copies of ourselves in this picture. So it provides, I think, one way and maybe at this point, maybe the only appealing way, in my opinion, to resolve these inconsistencies with quantum mechanics, although the mileage may vary and I don't want to speak for everyone by any means. This extends. That to me is already something. And the second thing is that it provides a picture. Quantum mechanics, when I teach quantum mechanics to students, they expect me to talk about waves moving in three-dimensional space. And when I show them the axioms, many of them are quite disappointed. They're like, but where's the picture? Isn't it as if there's an electron and it emits a photon? And I say, well, the textbook axioms don't really talk about an electron here emitting a photon there. According to the axioms, this is just added for color. We draw pictures of electrons emitting photons just to help tell the story. But none of that is actually sanctioned by the axioms of quantum mechanics. There's also the problem that there are lots of phenomena that happen in the world that are not measurements. You know, when early gases mix in the early universe, when distant stars collide, when a piece of aluminum exists on the surface of Mars and doesn't collapse under its own weight, when birds forage, when humans fall in love. These don't seem to be measurements. Yet the axioms of quantum theory only make predictions about measurement outcomes, probabilities of measurement outcomes, averages of measurement outcomes weighted by the probabilities of measurement outcomes. That's it. That's a very narrow class of phenomena. How do we go from that narrow class of phenomena to the broader class of all other things that we think happen in nature? Or are we just going to say that our talk about all those other things, about birds foraging and early gas mixing, is just for color, just storytelling? And this allows quantum mechanics to be a theory of a much larger class of phenomena, not just a narrow class of measurements. I call this the class problem in textbook quantum theory. But ultimately, I think physicists will be interested in applications, practical applications. So—or you were hoping. Well, we were hoping, but yes. But in my experience, physicists, you know, want to know, what can we

What should we do with this? I think there are some things you can do with this. The first is that indecomposable random processes, at least classical indecomposable random processes, are currently three years old. This is a completely unexplored area of dynamical systems theory. It’s like opening a paint-filled palette and flipping the page and finding a completely blank new page. How often does that happen in mathematics or science, that you flip the page and open up a whole new area of research? I don't know much about the mathematics of indecomposable random processes because nobody knows it yet. I think that’s interesting. I think it would be very exciting for people to think about these kinds of systems and discover their properties.

Perhaps the best way to study an indecomposable random system is to write down its Hilbert space and essentially do quantum mechanics. Just as sometimes the easiest way to study a Newtonian system is to write down its Lagrangian and do Lagrangian mechanics. Maybe we can understand things about these systems according to their own merits. Maybe when you don’t start from Hilbert space, but you start with these different axioms, you can point to new ways of generalizing quantum mechanics that were impossible to imagine before. If you start with Hilbert space, it limits your imagination about what you can do too. If you don’t start with Hilbert space, you’ll start with a more general kind of probabilistic system as I was talking about in this picture, this indecomposable random process. Maybe there are deformations you can make to the assumptions, changes you can make to generalize these systems in ways that don’t give you Hilbert space. You can generalize these systems and get something else that doesn’t give you Hilbert space which you could never have gotten to if you had started from Hilbert spaces. And maybe we need such a generalization in order to make progress on fundamental questions related to quantum gravity. So I think that’s another potential area of application.

And there's another area, which is that if there is this correspondence between Hilbert space representations of quantum systems and classically looking probabilistic systems, especially beyond the Markov approximation, this might point to efficient ways of simulating classical non-Markovian random systems using quantum computers. So one of the open questions is if we had a quantum computer today, what would we use it for? There are already a number of known algorithms. There's no proof that the algorithms are more powerful than what you can do classically, but it is widely believed that many things you can do with these quantum algorithms are far more efficient than you can do with classical algorithms. But it's not as if every question you want to do on a classical computer can be done more efficiently or faster on a quantum computer. So knowing what kinds of things a quantum computer will help us do more efficiently is a really open question. And when you establish a correspondence between quantum systems and a large class of random systems, you open the door that there are some problems that might be hard to simulate classically that are easier to simulate quantum mechanically. I don’t quite know what those are yet. And this is an area of research that I’m working on now. But I’m talking to people who are working in the quantum simulation theory of classical random processes and we are trying to determine perhaps there are ways of using these correspondences to provide more efficient ways of simulating these systems and provide a new application for quantum computers.

Now, there's also the possibility that in some circumstances, indecomposable random systems might behave differently than quantum systems, at least textbook quantum systems. Perhaps there are slightly different predictions concerning the Wigner's friend thought experiment. Unfortunately, doing Wigner's friend thought experiment is probably beyond our experimental capabilities for the next thousand years. Who knows? But there are some cases where you want to model a quantum system, and you need to make extra assumptions in order to make actual predictions, assumptions beyond textbook axioms. For example, if you want to design a very complicated quantum system, you might have to add some extra axioms about, you know, what are the typical initial states of the system, and what kind of typical evolution do we expect. These go beyond textbook axioms. And it's possible that when you start with different axioms, you're actually modifying some of those assumptions. And so for some classes of very complicated quantum systems, it's possible that you will get slightly different predictions. But these are all very speculative claims. There's a lot of work to be done to determine whether any of this can work and when. So I want to be very clear, we are just at the beginning here. But even the existence of these correspondences at all, I think opens a lot of doors. And there’s nothing more exciting to me in science or philosophy than open doors. Okay, this is certainly cutting-edge, like the most recently developed in the last two years. And also, it might be widely applicable soon and might already be on the right track. Does it offer any insights into QFT, which relies on Fock space, which relies on Hilbert spaces? Yes, so that's an excellent question. So quantum field theories fall within the axioms of textbook quantum mechanics. Those axioms I mentioned, and Hilbert spaces and all that stuff. A quantum field is a particular kind of system that lives within the framework defined by those axioms. What makes quantum fields, well, there are many things that make them complicated. The single thing that makes them complicated is that they have an infinite number of moving parts. Unlike a system that consists of five particles, they have an infinite number of moving parts. We see that these systems have infinitely many degrees of freedom. And that causes all kinds of mathematical complications. When working with quantum field theory, like the way ordinary physicists use quantum field theories today, you often have to regularize the theory. I mentioned Planck introducing Planck’s constant for regularization. Yes. So the regularization schemes that we use temporarily make many of these infinities finite. So that we can do the calculation. And then what we do is we try to send the regulator to infinity. And usually we can do that. Then it goes away. And we believe that we have a good, robust prediction that doesn't depend on the regulator. Unlike Planck’s constant, the regulator goes away and we think we have a robust result that holds in the infinite connected limit. And there’s a beautiful and complicated story called renormalization and effective field theory that’s all about how to do this in a mathematically consistent way, in a reasonably consistent way, and generate good predictions. I won’t have time to go into all of that. But ultimately, it's true, this is still just a quantum system, a very complicated quantum system. And insofar as you can generate the axioms for any quantum system, you can generate them for quantum fields starting from an indecomposable random process. In this picture, the quantum field is a bunch of non-localized entities, spread out in space in configurations. And these configurations evolve in this indecomposable random way that can be modeled in the language of Hilbert space. In a sense, it’s like any other kind of system, you’re still going to face the same mathematical complications, and perhaps new complications because of the fact that we have an infinite number of complications and you have to do mathematical tricks to make these things finite. But yes, the answer is quantum fields fit into this framework. But it will inherently be very complicated because quantum fields are extremely complicated.

And does it reproduce the exact assumptions of quantum mechanics? Or are there some deviations that there's an experimental way of testing whether the random approach is the more correct approach? That's a good question. So at the level of the correspondence, there's just a map that takes you from the constituents of the indecomposable random theory to the corresponding Hilbert space description of the quantum Hilbert space representation. There are principle deviations because collapse doesn’t fundamentally happen in this picture. However, the deviations would be roughly similar to any no-collapse formulation of quantum mechanics. So Bohmian mechanics, de Broglie-Bohm pilot-wave theory where particles are guided around by wave functions, ready always in quantum mechanics, you know, many-worlds interpretation, these are all non-collapse interpretations. Collapse doesn’t fundamentally happen. There’s no axiom of collapse. And so you're going to get small discrepancies in the predictions of these theories compared to textbook quantum theory. And I can make this very explicit. Go back to Wigner's friend. According to textbook quantum theory, Wigner's friend inside performs a measurement that causes the quantum system inside the box to collapse. That system has collapsed. It’s collapsed for everybody. It’s collapsed for Wigner’s friend in the box. It’s collapsed for Wigner outside the box. It just collapsed. In a no-collapse approach, like many-worlds or Bohmian mechanics or my approach, there’s no actual collapse that happened. And what that means is that the wave function that Wigner uses on the outside, the quantum state is a non-collapsed quantum state. It contains super-position interference phase effects that haven't been erased. They haven't been erased because there wasn’t a collapse. So in principle, Wigner on the outside, using a sufficiently powerful measuring device, could detect those small discrepancies, which might be inconsistent with textbook quantum mechanics. But detecting them would be extremely difficult. So there's a paper from a few years ago, Scott Aaronson and Lenny Susskind and myself and I believe one other author where we showed what people call quantum necromancy, which just means that if Wigner could design a sufficiently sophisticated and sensitive experimental apparatus to detect those small discrepancies between a no-collapse interpretation and textbook quantum theory, then that apparatus with some relatively minor modifications could take a long-dead cat and bring it back to life, which we usually consider a thermodynamically impossible feat, right? That’s what they call necromancy, like magically bringing the dead back to life. It would be more akin to unscrambling an egg, right? Like taking a scrambled egg and unscrambling it back to the original egg. I mean that really goes beyond any technological capability we can imagine. So yes, it does make somewhat different predictions, but only insofar as other no-collapse interpretations make the same slightly discrepant predictions. And unfortunately, those discrepant predictions are so minuscule that they're extremely difficult to measure. Measuring them might be easier than measuring the black hole information problem. But kind of on the same level of extremely tiny suppressed effects.

Okay, before we end, I want to know what is a particle from your perspective? And also is your view called an interpretation? Is it a theory? Is it a correspondence? What do you call it? Why don't we settle that now? I don’t have a good answer to that question. I think you can call it whatever you want. One can think of it as a physical theory. It has variables and constituents in it, and those constituents have laws. So it’s a physical theory. It’s a physical theory that makes the same predictions as quantum theory in all realistic scenarios to my knowledge. You can think of it as an interpretation of quantum theory, in the sense that if you start with the formalism of quantum mechanics and ask, how do we interpret things? What do things mean? This provides a platform to actually talk about what’s really out there in nature. I should say it's not a one-to-one picture. A given quantum system might represent many different indecomposable random processes just as the Lagrangian, or better yet, the Hamiltonian formulation of classical physics can describe many different Newtonian systems. There's no one-to-one relationship between these pictures. You can think of it as a formalism because it's a new mathematical framework. I think what you can't call it is a recipe or a prescription because it's more than just a list of steps to follow. It's more than just an algorithm, just as the axioms of quantum theory are more than just an algorithm to follow. It actually tells a story about the things that happen. I don’t know if that answers your question. Do you want to follow up on that or do you want me to talk about particles?

What is a particle? Classically, of course, a particle is a point-like entity characterized by some kind of mass. The mass could be zero. I guess it’s a tachyon. It could have some strange imaginary mass. It’s characterized by spatial degrees of freedom, which means it has a location in some conception of physical space, typically X, Y, Z coordinates when I think about it that way. It will generally have different locations, at different times, which we can think of as like a trajectory. If it’s a truly point-like particle, it has no internal structure. It has no size. However, it can carry certain other quantities. It can have other attributes like electrical charge, which is related to how it interacts with fields and other kinds of particles. It can have some intrinsic form of angular momentum. You can have kinetic energy, but kinetic energy can sort of freeze out and become a kind of rest energy, like the internal energy of the particle. Angular momentum is a kind of orbital momentum, it’s a kind of spinning momentum, but you can also have frozen orbital momentum. And that's called intrinsic spin. Particles can carry this intrinsic spin which can be converted into other forms of angular momentum, just as rest mass can be converted into other forms of energy. That’s what a particle is. It’s a thing with this set of potential properties. In quantum mechanics, at least since Wigner’s work on representation theory, particles have been understood as the simplest kinds of quantum systems that have well-defined nature or behavior under spacetime transformations. If you ask what is the simplest kind of quantum system whose Hilbert space is as small as possible, it doesn’t involve anything fancy, but when we move in space or we rotate or we change inertial reference frame, we change the frame of motion, we just get a new element from the same Hilbert space. We don't get something new, but it's at its minimal. It’s irreducible. Then you have a problem in mathematics. What are the irreducible unitary representations, which just means they’re represented by certain kinds of operators in Hilbert space, representation of the spacetime symmetry group, the Poincaré group, the abstract representation of moving in space, changing the frame of motion, rotating. That’s a problem in mathematics. You can classify them, and what you find is that there are distinct kinds of Hilbert spaces with these properties, characterized by mass and spin and charges. They have kinematic degrees of freedom and so on. That’s usually how we think about particles in quantum mechanics. That’s all we can say in quantum mechanics because all we have are Hilbert spaces in textbook formulation. If you’re going to describe a system of particles, you have a Hilbert space. All we can do is talk about what that Hilbert space is, what its structure is, and the structure of the particle is tied to these transformations in spacetime. In the picture I’m proposing, particles go back to the classical picture. A particle is a point-like object in space, in three-dimensional physical space if we believe physical space is three-dimensional. With different properties or dispositions that interact with other particles. Dispositions? Yes. That means that when you interact with a particle, it might be disposed to give you certain results on your measuring apparatus, that’s all I mean. If you measure its position, you'll read what its position is, but if you measure some more complicated features, what you get on your measuring apparatus will be a more complicated result of interaction with the particle. It has a disposition to produce certain kinds of behavior. In metaphysical talk, we can say that some properties are categorical. They’re explicitly there as they are, and others are dispositional. That is, you look at it and what you see is actually like how the particle is disposed to show you what it will do. But ultimately, particles are pretty much what they are in the classical picture. Just point-like entities with some properties, different locations across time. So that’s all a particle is according to this picture. And I should say that I’m not saying that particles are the fundamental ontology of nature. I don’t know what the fundamental ontology is. But so far in your theory, what is the fundamental ontology? We go back to the old classical way of doing things, which is that if you want to model a certain system, you propose a certain ontology for it. You have to say what you think the configurations are. If you want to model particles, it would be arrangements of particles. If you want to model fields, it will be non-local field strengths distributed across space. If you want to model strings, if you want to model bits on a computer register, you choose the right ontology for the right kind of system and then you model it. It's possible that there’s some truly fundamental ontology of all of nature. I don’t know what that is. I think for some people, especially people who work in Everettian quantum theory, the idea that the fundamental ontology is state vectors or wave functions in Hilbert space is a very appealing idea. That is, we already know what the fundamental substrate of all of reality is. Hilbert space, there’s some Hilbert space and there’s an object or objects in Hilbert space and that’s the fundamental ontology of everything. I’m not saying that. Hilbert spaces are not fundamental in this picture. We go back to needing to be somewhat metaphysically modest. We know about some systems. Some systems look like particles, some look like fields. Perhaps particles are emergent from fields and we don’t need particles as a separate category. Perhaps fields are fundamental, perhaps they’re not. I think it's premature to try to guess what the fundamental ontology is at this point, and in this picture you don’t have to. All you have to do is pick the ontology that you need for the model that you want to talk about, particles, fields, registers on a computer, and then proceed from there. Maybe someday we will discover that there’s one ontology one model from which we can derive all the other models. That would be a very exciting day, but I don’t think we can guess what that is today and I don’t think we can just say it’s Hilbert spaces and be done with it. I think that’s premature.

There’s much more I’d like to talk to you about. And we haven’t talked about causality either. There’s a whole other connection to causality. Reichenbach’s common cause. Right, we haven’t talked about Reichenbach’s common cause principle. And that’s another reason why I think this work is potentially exciting because it has some connections to causality that I’m very excited about. I think we should explore that next time. Yes, perhaps we should talk again. Professor, it was great seeing you in person. I’ve been following you, again, politely following you. Diplomatically, whatever. You understand. For quite some time now. Why don’t we end, I know you’re about to run off to see your students, why don’t we end with your advice to students? Oh. Actually, two. Your advice to the students who are watching and the researchers who are watching. I will just use my fallback advice, which I think has been really useful for me. People have told me this and maybe I’ll share it. We all come to the work we do with a distinct set of strengths and weaknesses, just like I just said. You know, every person is a certain set of strengths and weaknesses we’re kind of like a point in a very high-dimensional configuration space or parameter space. So we're not ranked. You can’t rank people. We’re not a ranked set. It’s very easy for students studying science to think that they’re on a linear ranking system. There are people better or smarter than them. And if they’re not as smart as those people, then they won’t be able to make contributions, they won’t be able to do good work. And in my experience, having worked with many students over the years, but also having read a lot of history and the history of science, we’ve discovered that it's really true that we’re all, we’re all points in some very high-dimensional parameter space we’re not ranked. Knowing how to leverage your unique profile of strengths and weaknesses to make progress in work that you find meaningful, knowing how to do that is part of what it means to enter the academic workforce and grow and learn. And develop. So I would say don't worry so much about where you think you fall in the ranking and focus more on knowing how to bring your unique profile to the table and how to use that to make progress on things that you find meaningful. That’s maybe my best advice. And for the researchers who are watching, not advice, but any message you want to put out there, well, you can give advice too. Interdisciplinarity is great. We should have more focus and spend more money on places where disciplines intersect. I work at the intersection of physics and mathematics and philosophy and now I think, you know, statistics, you know, dynamics. There are many different areas that I meet in the place where I’m currently working. And I don’t think that’s new, but I think some of the most exciting work that’s happening in scholarship generally and certainly in science or philosophy specifically is happening at the boundaries of multiple disciplines. So I would say that maybe not for the researchers, but for the people who fund the researchers. Yes, spending more time thinking about how to foster work at the boundaries of disciplines where there's real cross-pollination between disciplines, I think would pay huge dividends for the advancement of all our work.

I agree. Last night, one of the reasons I’m here besides seeing you was at MIT, there was an event by Adi here yesterday, last night, called “ekkolapto,” which was about neuroscience, cognition, biology, physics, and language. Yes, it was a mixture of all kinds of major areas in science and technology and engineering and mathematics (STEM) and the arts where they intersect and can be used to create something really useful. But you have to be rigorous about it. You can’t just, you know. So that was fun. Anyway, thank you. It was my pleasure and I know you’re eager to move on and I appreciate that your students have been waiting. That’s great. I appreciate your time. Thanks for having me. That’s great. I really appreciate the invitation and all the work that you’ve done in this amazing podcast series. It’s amazing work talking to the audience and talking to people who are working in all these different fields about their work and clarifying complicated topics for everybody, I think is amazing work. Thank you. And next time the interview we’ll do will be more technical. We can even use the blackboard. We’ll see. We’ll see. Anyway, it was a joy. Thank you very much for the invitation. Thank you. And I also want to thank our partner, the Economist. I also discovered last year that outbound links are extremely important in the algorithm, meaning that the more you share on Twitter, for example on Facebook or even on Reddit, et cetera, it signals to YouTube, hey, people are talking about this content outside of YouTube which in turn significantly helps with distribution on YouTube. Second, there's a remarkably active Discord and subreddit for Theories of Everything where people explain their toes. They respectfully disagree about the theories and build as a community our TOE links for both in the description. Third, you should know that this podcast is on iTunes. It’s on Spotify. It’s on all the audio platforms. All you have to do is type Theories of Everything and you’ll find it. I personally benefit from re-watching lectures and podcasts. I also read in the comments that TOE listeners also benefit from replays. So please consider re-listening instead on those platforms like iTunes and Spotify and Google Podcasts, or whatever podcast app you use. And finally, if you’d like to support more conversations like these, and more content like this, please consider visiting patreon.com and drop CurtJaimungal and donate whatever you want. There’s also PayPal. There’s also crypto. There’s also just joining on YouTube. Again, keep in mind that support from patrons and from you allows me to work on TOE full time. You also get early access to bonus episodes whether it’s audio or video. It’s audio in the case of Patreon, and video in the case of YouTube. For example, this episode you’re listening to right now was released a few days ago. Every dollar helps far more than you think. And in either case, your viewership is kind enough. Thank you very much.