There’s No Wave Function? | Jacob Barandes [Part 1]

I didn’t know it was impossible. Therefore, I approximated these theories to each other, anticipating that there would be a moment where I would need to make some kind of leap to transition from one theory, classical physics, to the other, quantum physics. And the strangest thing happened. The faces of the abyss merged. And there was no longer an abyss.

Today, I have a special treat. I traveled to Harvard University to meet Jacob Barandes in person, the co-director of graduate studies in physics and the person who has created a stir at the intersection of fundamental physics and philosophy. In his modest office, he was working on a revolutionary framework that finally explains quantum mechanics. His radical proposal suggests that there is no fundamental wave function, that Hilbert spaces are not real, and there is no mysterious quantum world. It’s just a mathematical convenience. It’s a dependent of the theory. But there’s nothing in physical reality that describes it. In this episode, we delve into the picture that solves the measurement problem, eliminates the need for multiple worlds, and finally gives a clear view of what’s actually happening at the smallest scales of nature. However, this clarity comes at a price. And the question is: Are physicists ready to accept quantum mechanics without magic?

Professor Jacob Barandes, welcome to Theories of Everything. I’ve been following you for a while, stalking you online in a non-creepy way, and we’ve been talking off-air about physics. However, you’re also the co-director of graduate studies here in physics, and you have a fascinating background in terms of how you got into physics. So why don’t we start there? Why don’t we start at the beginning?

From the very beginning, my first intellectual passion was the philosophy of mind. When I was five years old, I was sitting in one of those little gathering rooms that you have in schools, you know, like my elementary school, and one of the teachers was talking loudly about something. I don’t remember exactly what they were talking about, but I remember sitting on this soft carpet, and there was a piano, you know, the piano legs looked like lion’s feet. And suddenly I became aware of my existence in a very clear and explicit way. And I was deeply puzzled, and I tried to express my state of puzzlement to one of the other children sitting next to me, and I utterly failed to be able to explain what I was feeling in that moment. And so this left a kind of mark on me. And from that moment on, I was very interested in questions related to, you know, what is the nature of our existence? What does it mean to be conscious? So the philosophy of mind is where things started for me. But of course, I didn’t have a language to express any of these ideas. I loved science documentaries. I grew up in New York City, and my parents would take me to the American Museum of Natural History in New York City, which is a truly wonderful, magical place. And anyone who goes to New York City should really visit it. It’s amazing. So I became really interested in science. I became interested in mathematics. And I thought that if you’re interested in science and mathematics, and you’re interested in foundational questions, that means you should go into physics, right? That’s where I figured mathematics and these foundational questions, where they come together. So I went, you know, through school. I got to high school. I first studied physics in an academic way in high school. I loved it. It was my favorite class in high school. I still have all my old notebooks from that time. It was the only class where I was excited to go home and do homework, and that was great. My teacher, Doug Bartell, was exceptional. He was just amazing. Around that time, I also started doing return trips to New York City. We had moved out of the city at that point, about an hour north. And so I would take the train and go into the city on weekends. And I would take science classes in the city. Columbia University has a program called the Science Honors Program. At the time, it was run by Alan Barr, who’s a wonderful human being who was a physics professor there and also ran this special program, the Science Honors Program. And these were classes you could go to. There’s no homework or grades or tests. You just come on a Saturday morning, and you will learn all kinds of amazing things. It was an amazing program. And I would also work at the American Museum of Natural History. So I started interning there when I was in high school. And I started working in the astrophysics department. I wasn’t an expert in astrophysics in high school, but I was very curious about it. And I was doing a lot of computer programming at the time. And they were interested in having me work on some of the projects that they were doing. They were working on computer graphics visualizations of the galaxy. I worked on a project called the Digital Galaxy Project, if I remember the name correctly. And that was my first opportunity to work with scientists. And I didn’t know any scientists. We didn’t have any scientists in our family. My father is a physician. But research scientists, I had never met anyone like that. I didn’t know academics or professors or anything like that. So these two experiences were very important for me. And with the physics class, I decided that I wanted to study science and maybe physics. But my love for the philosophy of mind was still very strong. And so when I went to college, I decided that I was going to try to study the brain. I loved computers. And so the idea was that I would somehow find a way to combine my interest in computers and neuroscience somehow. And in order to move in that direction, I had to take some basic science courses, including physics, and I had to take math courses. And I really fell in love with those classes. I started studying biology in my sophomore year. And I enjoyed it, but not nearly as much. And I have a lot of respect for people who work in biology, but this wasn’t for me.

Yes. For me, I just want to know if the reason is the same for you. For me, it’s just a multiplicity of terms. And I feel that it’s memorization rather than derivation.

Yes, I sometimes feel that way. I mean I want to make it clear that I’m not a biologist. And I don’t want to, you know, make any assumptions about the way biologists think about their subject matter. I know that there is an internal logic to biology.

Oh, I just mean that from the freshman level, at least from what I went through, the older I get, or the more experienced I become in research, maybe it’s different. I went through this experience when I took the first semester of biology, and it was microbiology, and it was very logical. Then the second semester was anatomy. And that was much more memorization. And that I went through the same experience that you describe, it wasn’t really for me. You know, but it was hard to convince myself that I should study physics because, you know, there were all these incredibly smart young people studying physics and mathematics, and I was interested in the theoretical side, the mathematical side of physics. And I didn’t feel like I was good enough to keep up with people. But in my sophomore year of college, you know, I had conversations with some really wonderful people, some of my fellow college students. And basically, I decided, you know, that it’s really important to do what you’re excited about, and then, you know, you want to be the best at something. So I’ll do, I’ll do that anyway. And so I just started taking a lot of physics and math courses. And it was great. It was an amazing experience. And some of the philosophy courses that we were required to take at the time didn’t really appeal to me. There was a heavy focus on ancient Greek philosophy and early modern philosophy. Now, I appreciate those subjects, but, you know, they weren’t as exciting to me as what I was doing in physics, at least at that time. Now, I should say that, you know, as the years went by, and I started exploring other areas of philosophy that weren’t part of the standard college curriculum, analytic philosophy, philosophy of science, logic, foundations of mathematics, philosophy in physics, my views on philosophy changed completely. But, you know, at the time, I wasn’t, I didn’t have the opportunity to see a lot of that. And I didn’t see it grow up. And so I figured, okay, I’m going to focus my attention very much on physics. So I finished my undergraduate degree, and then I went on to graduate school. And I started my PhD here at Harvard University in theoretical physics. And as I progressed through that degree, you know, I started thinking again about some of these old philosophical questions that I had. And I did my PhD in high-energy theoretical physics. And, you know, I think we did some really useful work. But I was actually more excited about the philosophical questions. And by the time I finished my PhD, I knew that this was the direction I wanted to go in. So I continued working in the department. I’m a lecturer here in the department, and I also co-direct the graduate program as I think we talked about. So, you know, I’m mixing teaching physics, research, and advising students. It’s been a really rewarding experience. The students here are amazing. They’re ideal, and they’re great. And they’re profound. And I’ve really met the most wonderful people anyone could ask to meet here. I’ve met lifelong friends here. I mean, it’s just been an amazing experience to be here. And I think I’ve learned a lot, much more than any of them have learned from me. But that’s how one who works primarily in philosophy, philosophy of physics, finds oneself embedded in the physics department. So I have a faculty affiliate with the philosophy department, and I rotate teaching physics classes there as well.

Yes. But my research is primarily around the intersection of philosophy and physics. So what happened to the philosophy of mind?

That’s a good question. I’m still really interested in the philosophy of mind. I don’t think that my set of tools is what I think, you know, we all come into this world with a distinct profile of strengths and weaknesses. I’m very interested in the philosophy of mind, but I don’t think my distinct profile of strengths and weaknesses is optimal for working in the philosophy of mind. So I read about it. I’m intensely interested in it. But I feel I’m better suited to the questions, the intersection of physics and philosophy. And that’s where I’ve spent my, I’ve spent my time and my focus. You said earlier that you don’t want to pursue something just to be the best at it, you want to pursue your interests. But you’re saying here, well, my interests still lie in the philosophy of mind, but it’s not that, I’m not the sharpest tool in the shed when it comes to that. That’s not my skill set. How do you reconcile those?

That’s good. Yes. I mean we all change as we get older, right?

Yes. So I think for me it’s been about finding balance. It’s less important to me. And I, you know, I’ve talked to students about this as well, right? It’s less important, you know, what you want to be. It’s not important to be big or famous or important. And for me, what’s most important is to do meaningful work, which I feel is meaningful, which the person feels, is meaningful work which we feel is meaningful, but we also feel like we’re making progress. So, it’s not that I don’t want to work in philosophy of mind because I think that would not make me famous. My lack of motivation to work in the philosophy of mind, at least right now, and this could change, is because I at least right now, don’t feel that’s where I’ll be most productive. It’s where I feel that I won’t feel like I’ll do work that I felt proud of like the work that I do in philosophy of physics and philosophy of science. And that’s why primarily I focus on these areas now.

I’d like to get you to justify the non-existence of a wave function shortly. However, I want to know if there’s a relationship between the philosophy of mind and quantum mechanics first.

Yes, that’s a very interesting question. And in fact, this will lead us to a very important research paper which I hope we can talk about from the early 1960s. It was a paper written by Eugene Wigner. I think the paper was called The Question of Mind and Body. The title might be slightly off. But Wigner presented a thought experiment that I think is in some ways related to his view that there’s something connecting consciousness and the collapse of wave functions. This thought experiment actually goes back earlier. It’s not entirely clear to me who originated this thought experiment because it first appeared in Hugh Everett’s extended dissertation. Hugh Everett was a graduate student at Princeton University and was at Princeton at the same time overlapping with Eugene Wigner. And Hugh Everett originated the many-worlds interpretation of quantum mechanics and wrote this very long dissertation. It’s 137 pages. I think that’s maybe not long in absolute terms. And in 1956-1957, he didn’t publish it in the end. His advisor strongly encouraged him to publish a different, shorter dissertation. But you can find a copy of this extended dissertation and it opens with the same thought experiment. So we know that this thought experiment was presented before Wigner by at least seven years before Wigner’s paper or something like that. But that wasn’t published. Wigner’s paper was published and so it’s widely attributed to him. Wigner did this thought experiment which I’ll talk about in more detail if you like because I think it gives a good motivation for why one might think there’s a problem with the way we formulate quantum mechanics at the textbook axiomatic level. So I’ll come back to it. But I’ll just say that this thought experiment was seemingly connected to Wigner’s idea that consciousness is somehow connected to the collapse of wave functions. And I think this creates a potential connection between philosophy of mind and quantum mechanics. But there are other connections. In the many-worlds interpretation, which is a formulation of quantum mechanics presented by Hugh Everett in the mid-1950s, one faces some fundamental questions about consciousness and human identity in a universe where a gigantic universal wave function splits into parallel realities, each of which contains copies of yourself. How do we understand that kind of picture and make it related to the way we think about fundamental problems in the philosophy of mind? I don’t claim to have an answer to that question. I think it’s an ongoing puzzle. So I think there are some interesting connections between philosophy of mind and quantum mechanics. And I think my general position has been that anytime you’re unable to answer a question about quantum mechanics, because you’ve somehow related it to the philosophy of mind, and in particular to an unanswered and perhaps unanswerable problem about the philosophy of mind, I’m deeply concerned.

Okay, let’s talk about the wave function, because when we talk about the collapse of the wave function, the collapse of something, what is the wave function? And why do you think there is no such thing?

That’s a great question. So let me say a little bit about some history. I think some history here might be helpful in setting things up. But before I talk about the history, let me first answer your question. I’ll give a short answer, and then we’ll talk a little bit about history, otherwise we might get lost in the weeds. There’s a common story about quantum mechanics that a lot of students coming into this field hear. The first fundamental object in quantum mechanics is the wave function. It’s this mysterious wavy object. It’s somewhat unusual. It’s kind of like the electric field, except its values are complex values. They’re complex numbers. They’re like regular real numbers plus a number multiplied by the square root of minus one. And somehow, they’re related to probabilities somehow. And somehow, the goal of quantum mechanics is wave functions. They’re the central player in the story. And the way you think about quantum mechanics starts from the way you think about wave functions, right? And they are the heroes of this story. I think there are some good reasons to be a little skeptical of that picture. And I don’t think there’s much really—there’ll be a lot of debate about saying—I think, you know, as you learn about quantum mechanics and you start to see that there’s a much bigger picture, you know, you start to see that the wave function in this kind of story is an oversimplification. I think if you pressed a person working in physics hard enough, they would admit, actually, that this picture is a little bit—it’s a little bit quick, and there’s a deeper picture underneath. And so, I think one perspective on what I’m saying here, your question, is to take those concerns and take them to their logical conclusions, right? We ask ourselves, well, what is this theory really about? Is it really about wave functions at all? And in that regard, are wave functions the things that describe things that actually exist in the world? And this is where one lives in a kind of borderline space between physics and philosophy when you ask questions like this. On the one hand, it’s about physics. It’s about, you know, what’s the right way to think, and what’s the right way to formulate, you know, what can be our best and most successful scientific theories, and certainly when it comes to numerical accuracy and precision. You know, and on the other hand, what’s really out there? What’s in the world? What is existent, right? That’s the ontological word that metaphysicians use. Ontology is the study of what truly exists. And so let’s ask, does the wave function describe something physically existent? That’s a question. To explain a little bit about how we get here, I can say a little bit about history, but let me pause and ask, do you have any follow-up questions on that? I want to be careful, you know, to make sure we all have time for that.

Certainly, certainly. So when you say there’s the standard picture of the wave function, you said, well, what people learn is that there’s a wave function, and it’s associated with complex numbers, and it’s also associated with probability somehow. Is that the picture you mean? Or are you referring to the picture that represents the wave function is something in physical space and it’s of something?

Yes. So when I say that, what I mean is, when people say the wave function, when physicists talk about the wave function, it’s the wave function of something, the wave function of the electron, the wave function of so-and-so. So what’s that picture that you’re about to break down specifically?

Yes, that’s an excellent question. I think, you know, the problem here is that it contains a little bit of both, right? The wave function plays a functional role in quantum mechanics. That is, it does certain things for you theoretically. And we study its change with time. There’s this famous equation called the Schrödinger equation, which roughly says that when the system is left to itself, it’s not disturbed, it doesn’t exchange information with any other systems. Then it has a wave function that changes with time in a smooth way. And this smooth process is described by a differential equation, which is roughly an equation that just tells you what the wave function is at every infinitesimally later moment, given what it was at the previous moment. And that equation is known as the Schrödinger equation. So we plug in the initial wave function, and we use the Schrödinger equation to find out what the wave function will be, and how it will behave as time passes. And then there are rules for taking the wave function and calculating predictions. So that’s not controversial, and I certainly agree with that picture, where we should use wave functions for that purpose. But I think it’s easy to let that picture lead one to believe that wave functions describe physical things that exist in the world. And, you know, when I think students first arrive at a quantum mechanics class without having studied the theory, they might think that, well, every particle has some wave function, and wave functions are kind of like, they’re kind of like waves. And, you know, empty space is kind of like the ocean, and all the particles are like little waves moving in the ocean. And that’s actually not the picture that we get even from textbook quantum mechanics. In textbook theory, the wave function is an assignment of a complex number to every possible configuration of your system. If you’re talking about one particle, nothing else, then every possible configuration is every possible location the particle can be. And each location is distinguished by three numbers. They’re the x, y, and z coordinates, if you like. And so the wave function takes those three coordinates and assigns them a complex number, and then we’re supposed to do a mathematical operation with that complex number, and the probability comes out. More precisely, the probability density, but the probability per unit volume that a measurement will find the particle in that spot. So it’s very easy to think that the wave function is like a field, like the electric field, in three-dimensional space. But if you have two particles instead of one particle, the probability space is more complicated. We call this probability space the configuration space. Because every possible configuration of a two-particle system requires six numbers, not three. You need to know x, y, and z of the first particle, and you need to know x, y, and z of the second particle. So the wave function assigns a complex number to a point in six-dimensional space. Six dimensions because you need six numbers to specify its points. That means that the two-particle wave function is a function, as we say, its domain is six-dimensional space. Six-dimensional space is not three-dimensional physical space. If you have three particles, it’s nine-dimensional space. If you have ten particles, it’s thirty-dimensional space. And in fact, our universe doesn’t have a well-defined number of particles at all. Our leading physical model of the universe, at least for the non-gravitational parts of the universe, is the standard model, which relies on a set of models known as quantum field theories. And in quantum field theories, particles are emergent excitations of these kinds of non-localized entities called quantum fields. And the number of particles can change from moment to moment. It’s not always well-defined. So it’s not even clear how to think about wave functions living in anything like physical space in the universe as we know it. So I don’t mean to say that we’re teaching our students wrong. We’re not. I mean students will learn this as they progress in their physics trajectories. But I think many people outside or new students who haven’t yet begun their own journey in physics have a certain idea about what wave functions are that’s actually quite different from the way we use them in practice. I don’t know if that clarifies what you were asking.

Okay, you have a lecture on how Hilbert spaces aren’t real as well. I’d like you to talk about that. Now, I understand you said more precisely that Hilbert spaces are superfluous in the same way that gauge symmetries are superfluous or that gauge transformations are superfluous. So at this point, if it’s okay, let me talk a little bit about history, if that’s okay.

So where does quantum theory begin? Blackbody radiation, like a grand tour of quantum theory. There are these actual physical things called blackbodies. They are rooms that are heated. And, you know, there’s a small hole in them. I mean you don’t really drill a hole in them. It’s more complicated. I’m very much a theorist, and my conception of experimental physics is that it should be better. And I apologize in advance to any experimentalist who is listening. But these were actual systems. People had actually built these things. They were rooms. They would heat them up. And then they would have a small hole in the room through which they could see radiation coming out of them. And they could put this radiation through experimental apparatus which could tell them how strong the radiation was as a function of wavelength. We all know that different wavelengths of light correspond to different colors in the visible range. And if you go outside the visible range, you have very long wavelengths infrared or, you know, microwave radiation or radio waves. And at very short wavelengths, you have X-rays and so on, gamma rays going out, and ultraviolet, and gamma rays go out to very short wavelengths. And you can ask yourself, if I plot how strong the radiation is as a function of wavelength, what kind of pattern should I expect to see? And it was difficult to explain the pattern that was revealed in experiments on the basis of a first-principles theory. A very important physicist, Max Planck, in 1900, discovered a way to generate a theoretical prediction for what the blackbody radiation curve should look like. And it agreed with experiments. But in order to get there, he had to somehow fudge his formulas. He had to introduce the fudge factor. He had to assume that the radiation in this room could only occur in quantized amounts. That is, there were different wavelengths you could produce, but each wavelength could only be excited in discrete steps. This led to…

The quantum hypothesis. This was his quantum hypothesis. There was a kind of parameter that told you how much you wanted to implement these steps. Today, we might call it the regulator. And this parameter, we now call it h, lowercase h. We call it Planck’s constant. Planck introduced it, and I think his idea at some point was that he wanted to introduce an intermediate step in the calculation, and once he calculated everything, he would send this clearly, in his mind, non-physical parameter to zero. But whenever he tried to send it to zero, he got wrong results. He realized that this parameter was not zero, but very small. It had, today, a value of the order 10 to the minus 34, and 10 to the minus 33 in conventional units, and he couldn’t get rid of it. And so people had to accept that there was this fundamental distinction in nature that he couldn’t explain.

Then over the next 22 or 23 years, the period from 1900 to 1922, 1923, physicists working on these questions lived in a time we now call old quantum theory, a model we call old quantum theory. Old quantum theory was based on a mixture of physical pictures of particles moving in space, in orbits, as in atoms, and then a set of ad hoc formulas and rules that people didn’t quite understand that seemed to capture some of the observations that were coming out of experiment. But it was a very murky time. There was nothing like a fundamental theory from which this whole picture emerged. And the formulas weren’t perfect, they didn’t work quite right.

In 1913, Bohr proposed, as in Niels Bohr proposed his famous model of the hydrogen atom. This is a model where you have the nucleus, and for the hydrogen atom it’s just a proton, a positively charged proton in the middle, and an electron orbiting it. By leveraging this kind of ad hoc formulas that were characteristic of the time, Bohr argued that the electron could only be in certain specific orbits, and when it jumps between them, its energy changes in discrete steps, and whenever its energy changes, it either absorbs or emits radiation in discrete amounts. And with this, he was able to explain the specific colors of radiation that we can see emitted from excited hydrogen atoms, or absorbed by hydrogen atoms, what are called line spectra. And we also found that other atoms had line spectra as well. Helium, famously, was discovered because we looked at the spectral lines of the sun, and after accounting for all the spectral lines we knew, there were still more lines that we hadn’t accounted for, and so people guessed that there was a new element. They called it helium after Helios the sun, and eventually we found helium on Earth, but it was first discovered in the sun. And clearly this was something very significant, and Bohr ended up winning a Nobel Prize in 1922 for this Bohr model.

But by around 1922, at least by that time, people became very skeptical, because people couldn’t quite capture these clear pictures of particles moving in specific ways, particles interacting with fields like the electric and magnetic field, which were well understood by this point. They couldn’t capture this picture of the world, this existence, this picture of what was physically there. They couldn’t find the laws that when combined with this picture gave you the correct predictions, that gave you sufficient empirical meaning subject to experiment, empirically sufficient, a theory that was able to explain the things we were seeing, and make predictions that were regularly confirmed. Niels Bohr gave a set of lectures in 1922, and Heisenberg, Werner Heisenberg attends one of these lectures, at least one of them, and the story goes on, and I’m not entirely sure how much of the story is made up or how much actually happened but the story goes that Heisenberg at some point objected to this lecture. Now Heisenberg was 20 years old, maybe 21 years old, and Niels Bohr was a Nobel Prize-winning physicist, widely regarded as the whisperer of quantum, somebody who intuitively understood quantum mechanics in a way that nobody else did. And here was Heisenberg challenging Bohr. And I think it was expected that people would think that Bohr would be upset about this. But the story goes that after the lecture, Heisenberg went on a long walk with Bohr, for many hours, and this had a profound effect on how Heisenberg thought about nature. The story goes, the legend says, that Bohr revealed to Heisenberg that, in Bohr’s words, he wasn’t convinced that there were particles or orbits at all. You start seeing comments from people like Sommerfeld, and Arnold Sommerfeld was actually Heisenberg’s PhD advisor in Munich where Heisenberg was. And then people like Wolfgang Pauli, many of the people who founded this developing theory were increasingly skeptical that this world picture could survive.

Then in 1925, Heisenberg, who was visiting the leading institution in mathematics and theoretical physics at the time, Göttingen, where there were people like David Hilbert, and there was Max Born, and Felix Klein, leading mathematicians and physicists. Heisenberg was visiting there, and he was visiting Max Born’s group, Max Born and his student Pascual Jordan. And he was thinking about all these questions and was very confused about them all. There were all these new more complicated formulas that people were writing down that were connecting energy levels and transitions and atoms to spectra. And he got this terrible case of hay fever. In the spring of 1925, he went to Helgoland, which is this island where the pollen levels are much lower. And he comes back with this utterly strange draft of a paper. And he gives this paper to Max Born and Jordan, and then goes on vacation again. He goes on some kind of vacation. He’s exhausted from all his intellectual efforts. And this paper begins with these remarkably striking statements, statements that a philosopher of science will immediately recognize as pointing to a kind of paradigm shift. Heisenberg begins by saying, we should not think of orbits or particles anymore. We should entirely reformulate our physics, our theories, in terms of quantities that are in principle measurable. And so he formulates this different way of thinking about quantum mechanics in terms of abstract objects that Max Born identified as matrices. People listening to this may have heard of a matrix. And the subject to which they belong, which is linear algebra, was not a fundamental part of the physics curriculum at the time. In fact, Heisenberg independently discovered matrices in this work. And their computational operations. And Born realized what Heisenberg was doing. Then together they wrote two papers where they presented this thing called matrix mechanics. In matrix mechanics, there is no longer a picture. There’s no picture of atoms and electrons and particles orbiting atoms. There’s just this abstract mathematical apparatus for predicting the energy levels of things.

Schrödinger comes along a few months later, and through a series of theoretical arguments that I won’t be able to do justice to here that connect to what’s called Hamilton-Jacobi theory, a beautiful area of classical physics. Heisenberg proposes a different way of thinking about quantum mechanics. He introduces his wave function. And as it appears in these papers. Schrödinger introduces his wave function. This was in early 1926. And in some papers in German and in English he wrote in both languages. He called this theory the wave theory of quantum mechanics, which is a beautiful word, isn’t it? Wave theory, it’s a great way, beautiful term. And he introduces his wave function, and as he works with it he treats it as a mechanical object. He immediately recognizes that it lives in configuration space. He says this in his early papers, but he says maybe this is reality. Maybe reality is this giant undulating mechanical wave in configuration space. And through some mechanism that we don’t quite understand, this undulating wave in this abstract high-dimensional space somehow projects facts into three-dimensional reality, facts about where the electrons are. He doesn’t have the full picture here, but this is essentially what he describes. And he’s able to use this wave function as an indirect way of predicting energy levels. Max Born comes along very shortly thereafter and suggests that wave functions are not mechanical objects. They are mathematical tools. The reason why they’re appreciated in a complicated way is that they’re not really physical things. They are things that we work with using this kind of mathematical operations to generate probabilities. Wave functions, when you take the complex numbers that describe them and perform this operation on them, what comes out are probabilities. The wave function should be understood as a mathematical apparatus for generating probabilities.

By 1928, Schrödinger had already started to back off his point of view. He gave a lecture in 1928, his fourth lecture on wave mechanics, where he said he used to think that wave functions were physical things. He even foreshadows, in some ways, the many-worlds interpretation. He says it’s an object where everything the system can do plays its role in this giant wave function. But he no longer believed this. He had accepted what seems to be the new idea. And then over the next couple of years, 1930, Paul Dirac wrote a textbook where he summarizes all of quantum mechanics that was known at the time, and he connected Schrödinger’s wave functions with Heisenberg and Born and Jordan’s matrix mechanics. And he realizes that they’re all part of this deeper reality. There’s this mathematical structure. It’s a kind of space called a Hilbert space, which is a kind of vector space. But it’s a vector space that contains complex numbers. And when you look at the vector space in one way, you see the wave functions. If you look at the vector space in another way, you see the matrices. You see what Heisenberg was doing. They are all parts of this mathematical structure that is called Hilbert space. And then two years later, John von Neumann, the great mathematician, wrote a book called the Mathematical Foundations of Quantum Mechanics, where he formulated this theory in rigorous mathematical terms. And that’s what we’ve had ever since.

Today, when people refer to the Dirac-von Neumann axioms or the axioms of quantum mechanics, they mean this recipe for generating predictions about what we’ll see in experiments, empirical predictions. And this framework relies on Hilbert spaces. Roughly, the first axiom says that every quantum system has this abstract space called Hilbert space, which is not configuration space. It’s a completely different kind of thing. And that the state, to some extent, of the quantum system is represented by certain kinds of objects in this Hilbert space. State vectors, in the simplest case, are density operators, somewhat more complicated. The second axiom is that when the system is left to itself, it evolves according to Schrödinger’s equation, generally, unitary evolution. Then there are three other axioms that talk about when a measurement is made on the system, and what you’re likely to see. These are what are called the measurement axioms. One of them concerns the kind of mathematical objects that represent things we can measure. These are called observables. One of them concerns how to generate probabilities for what you’ll see when you make measurements. This is called Born’s rule. And the last one is that after the measurement, the state vector of the quantum state collapses. This is the way we present quantum mechanics to students in textbooks now. If you go to my shelf and look at any of the textbooks on quantum mechanics, it will present something like these five axioms. Different people will sort them out differently. They might not say there are five. They might say there are four or six or some other number. But I like to divide them into five, okay?

So we know that we have these axioms. We know they’re very good at giving us a prescription for generating empirical predictions. But what do they tell us about what’s actually going on in nature? Do they say that the seat of reality is actually an abstract Hilbert space, or a very high-dimensional or infinite-dimensional vector space over the complex numbers? And that the universe is some object in that space that evolves according to some rule? There are some reasons to be skeptical about this. The first one is that we can reformulate the axioms of quantum theory in a completely different language. So people who work in different other areas of quantum physics, in mathematical physics, sometimes prefer to formulate quantum mechanics in terms of what are called C-star algebras. C-star algebras are not things that you find on the beach. They’re not that kind of C-star. But they’re abstract sets of mathematical entities that encode how systems evolve in quantum mechanics, and how to generate measurement predictions in a way that doesn’t begin with Hilbert spaces. When you take this alternative point of view, you get a very general mathematical framework from which the Hilbert space picture can emerge for sure, well, you can almost always pull out a Hilbert space picture in these stories. Sometimes you get a unique Hilbert space. Sometimes you get a proliferation of different Hilbert spaces, and there are all kinds of deep questions about what that means. But when I started learning about the algebraic formulation of quantum mechanics for quantum mechanics, it opened my mind, as I think it does for a lot of people, to the idea that maybe the Dirac-von Neumann axioms, the ones we hear about in textbooks, that are primitively formulated in terms of Hilbert spaces, may not be the end of the story, and that Hilbert spaces may not be the fundamental way of thinking about the structure of quantum mechanics. Now, I think one can go much further than that, and maybe we’ll talk a little bit about how one can go further. But once you actually start seeing that there are mathematically equivalent ways of formulating quantum mechanics that don’t start with Hilbert spaces, it makes you somewhat skeptical that the Hilbert space picture is fundamental, and that the objects in the Hilbert space picture… state vectors that become wave functions when viewed in a particular way, are fundamental objects.

Well, what allows you to say that Hilbert spaces that can come from the C-star way means that Hilbert spaces are more illusory? Does the C-star algebra allow you to calculate something that the Hilbert picture doesn’t allow, but it allows you to calculate everything that the Hilbert picture does? Does it then encompass it? So that’s an excellent question. In the standard approach to quantum mechanics, the first three measurement axioms say that everything observable you might want to look at is represented by a kind of matrix, and more technically, a self-adjoint operator in Hilbert space. It’s somewhat like an abstract mathematical entity that you can add other such entities to and multiply them together. It acts like a number, like the variable x in algebra, but it’s a little bit weirder, in that they don’t always commute with each other. They’re kind of strange objects, but we have a set of mathematical tools for taking them and generating empirical predictions. When you start with the textbook axioms, you start with Hilbert space, then these operators are the things that come out of Hilbert space. In the C-star algebra approach, you go in the opposite direction. You begin with an abstract specification of the things that are observable, which are represented by these abstract mathematical symbols. And the symbols have all kinds of mathematical properties, rules for how to add them together, multiply them, just as I said, but they didn’t start as things that live in Hilbert spaces. They have their own existence, their own identity. And there’s also a funny way that people who work in this area sometimes talk about it, where they’ll say, are you a Hilbert space conservative? Meaning you consider Hilbert space to be the starting point, or are you an algebraic imperialist? Do you consider the algebraic things to be the fundamental ingredients? When you start with these fundamental algebraic ingredients, they form a particular kind of mathematical structure called a C-star algebra. The C stands for closed, which means that there aren’t any holes in the algebra. Everything is filled in all the way through. And the star just refers to the complex conjugation operation, but on these objects, roughly. And there are some other properties that it needs to have, but they’re very primitive properties. And what you can then show is that there’s a way to build a Hilbert space out of these algebraic things. And once you’ve done that, they become the matrices or the operators in Hilbert space. But once you see this construction, it’s called the Gelfand-Naimark-Segal construction. It’s a beautiful theorem. Once you’ve done that, you start to wonder how important Hilbert spaces are, especially since some C-star algebras don’t generate a unique Hilbert space. And now you have this question of exactly which Hilbert space you’re supposed to use. And that’s where the Hilbert space proliferation comes from.

In a way, yes. So for simple systems, if you have one particle or three particles or 10 particles, it turns out that you basically get a unique Hilbert space. So you can really think of either one as being fundamental. But when you think about systems that have infinitely many moving parts, any system that you design as though it has infinitely many particles, we do this in thermodynamics, or in a quantum field where there are variables at every point in all of space. For that kind of system, you don’t necessarily get a unique Hilbert space. And then there’s this question that’s more fundamental. And this is an ongoing debate now. It’s called the problem of inequivalent unitary representations of C-star algebras. And it’s an ongoing puzzle in the foundations of mathematical quantum mechanics. Earlier, we were talking about Heisenberg and how he started with observables, and Schrödinger didn’t. And now you’re talking about taking those observables seriously and forming an algebra out of them only called a C-star algebra. But then you also said that you can take representations of those observables, which become matrices. Yes. And Heisenberg was dealing with matrices. Exactly. Right. Yes. Your picture is exactly right. So that’s not what I’m working on now. But when you start to realize that you can reformulate the rules of quantum theory in a different way, it kind of opens your mind to other possibilities. I want to know what it opened up for your mind. Yes. Where is your mind these days about quantum? That’s an excellent question. So let me go back to the Wigner’s friend thought experiment again, because I think that’s a good link to knowing where I end up, and where I think a lot of people end up. So what was this thought experiment about? In some ways, I mean, it’s a thought experiment that’s known to a lot of people in physics and certainly philosophy of physics. It’s a really good workhorse in quantum foundations. But I think there are still a lot of lessons that can be learned from it. And people have extended this thought experiment into many different areas. There are what are called the extended Wigner’s friend scenarios that a lot of people are working on. But I think the original version of it is really quite striking. And as I said, it was formulated back in 1956 in Hugh Everett’s original thesis. But it’s a very simple question. The axioms say we have Hilbert space and we represent the state of the system as some object in this Hilbert space. The questions that we can ask are represented by these observables. We can generate probabilities. Then after the measurement we collapse. Why do we collapse? What’s the point of collapse? Collapse is to ensure the strength of the measurement outcomes. After making the measurement, you are guaranteed to get the same result again. And that’s what the collapse accomplishes. It ensures that once you get a definite outcome, if you make the same measurement again before giving the system any chance to evolve further, you’re almost 100% guaranteed to get the same outcome. Now, the problem of course is, well, what is a measurement? What kinds of physical processes count as measurements? One point of view is that we don’t need to worry about this. One view is that all scientific theories are ultimately utilitarian projects, meaning that they exist only to allow us to connect preparations and setups of measurements and predictions to outcomes of measurements or measurement results. That’s all a scientific theory aims to do. To do anything else, to paint pictures of reality is just fluff. It’s like mythology. It’s not necessary. It’s philosophy. It’s metaphysics. It’s not important for physics to do that. And the theory does that. The theory says we make a measurement. And here’s what comes out. The theory does what it’s supposed to do. What more could you ask for? The theory says when we make a measurement, we do this thing, we apply this recipe, and we collapse the quantum state of the system, and then we move on. Everything works out fine. That’s your problem.

Wigner’s friend thought experiment considers observers. The first is Wigner, in the outside world, enjoying a nice sunny day, maybe a beautiful autumn day like this one. And Wigner’s friend who’s inside a completely sealed box. Now, people in quantum mechanics might wonder, can you really seal anything in a box? Can’t things get out? Well, particles can get out of certain kinds of containers via tunneling, but if you make the box thick enough, and impenetrable enough, in a very idealized sense, you can keep everything inside the box, at least for the duration of the experiment. Wigner’s friend is inside this box with a quantum system, some system in a superposition of two possibilities. Let’s say it’s a spin-up particle mixed with a spin-down particle in some superposition. Wigner’s friend engages in some interaction that we might call a measurement of this superposed quantum system that’s inside this box. Wigner, on the outside of the sealed box, doesn’t make a measurement, and he doesn’t in any way know what happened. Now we have two observers, and now we have a question. Do we activate the collapse postulate or do we not activate the collapse postulate? We have two observers, and now we have to take a stand, and we have to say whether we’re going to do it or not. And there are only a few possibilities. One possibility is yes, we activate it. We activate this measurement postulate, and the quantum state of the system collapses, even though Wigner on the outside hasn’t seen anything, it just collapses. For everybody, it just collapses. But now we have a question. Wigner on the outside, why does Wigner think it collapsed? Because Wigner’s friend made a measurement? But how do we know that the thing that Wigner’s friend did qualifies as a measurement? Wigner’s friend is just a physical system in the box. If you replace Wigner’s friend with an electron that somehow interacts with the other particle, we wouldn’t call that a measurement. If you replace Wigner’s friend with a tardigrade, a water bear, one of those microscopic highly resilient creatures that’s hard to see with the naked eye, would that have been a measurement? Now we just have a physics question. At what point do we cross the line to yes, measurement, or no, not a measurement? Now, there’s what’s called decoherence which we often hear about in this kind of discussion. Oh, when a large system interacts with a quantum system, there’s something called decoherence. Decoherence causes the quantum state of the system being observed to change. But decoherence doesn’t collapse anything. And it doesn’t single out one outcome. And you can actually prove that there’s no way to take the no-collapse postulates in quantum theory and get a definite outcome, to get one outcome singled out from all the others. So the question is, what do we do here? In principle, we need to give a precise definition of what kinds of processes count as measurements and which ones don’t, and that’s called the measurement problem. We don’t have such a rigorous definition, and we haven’t for a hundred years. Maybe somebody will come along someday and say, this is a measurement, and this is not a measurement, but at this point, we don’t have that phrase. And without it, we just have this ambiguity. This is the famous measurement problem. That’s one possibility. The next possibility is to say that there is a definite outcome of the experiment, but somehow it happens without collapse. Or that the collapse is perspective-dependent. Wigner’s friend sees the collapse, but Wigner on the outside doesn’t see it. But if you take the horn of this dilemma, the trilemma, the quadrilemma, and you go down this path, what you’re saying is that Wigner on the outside has a quantum state that he uses to describe the system, and this quantum state is incomplete. And he doesn’t know the definite outcome that happened in the experiment. And if you accept that the wave function is incomplete, then you accept that there are additional or hidden variables outside the wave function, which, again, doesn’t agree with the textbook formulation of quantum theory, at least according to the usual axioms.

The third possibility is to replace measurement with something else, some kind of dynamical process whereby things naturally collapse when they’re large enough, and there’s a whole family of approaches to quantum foundations called dynamical spontaneous collapse approaches that do that. The wave function or the quantum state is this thing, and for individual particles, it lasts for a very long time before it collapses, but when you put enough particles together, if any of them collapse, everything collapses, and this explains why it’s natural for very large objects to collapse very quickly. That’s the third option. These approaches are very interesting. Some of them have been tested experimentally. They involve a lot of new parameters, you know, and you have to guess the equations, and we don’t really have a well-motivated way of understanding which one should be used. The fourth possibility is that there’s no outcome. Nobody gets

On a specific measurement. Either there is no outcome at all, which doesn’t really seem logical, or all outcomes happen in some sense, which is what Hugh Everett argued in his thesis when he looked at this problem. There’s no collapse. We get rid of the collapse postulate, and Wigner’s friend splits into two parallel versions, each seeing a different outcome. One sees spin up. One can see spin down, and these are inside the box. Also, Wigner on the outside doesn’t register a definite outcome. If the box is opened, Wigner sees the definite outcome, and Wigner splits into two versions, and the universe kind of unravels with the copies spreading and branching, or the wave function of the universe splits into multiple possibilities. That is the many-worlds interpretation, and it is a source of ongoing contention. There are a lot of questions about how to understand basic questions about identity and the philosophy of mind in a world like that. It's not clear how to understand probability in a world like that. There are ongoing discussions about how to achieve that.

That is basically what we are left with. That's what Wigner's friend experiment left you with. You have to make a decision. I think there’s a fifth possibility, which is that quantum mechanics is wrong. And until we have a good reason to believe that this is just a mistake, maybe we should operate under the assumption that we need to make the theory work better. I think that’s a good motivator to take seriously the fact that something is wrong with the postulates of quantum mechanics, and that we need to do something about it. It’s not just that they are ad hoc, they are either vague or incomplete or inconsistent in some way, and we need to fix them.

Where does that leave me, and where do I enter? One way to understand what I’m trying to do is to go back to 1922 when people were willing to give up on the old pictures of particles and fields and actual things being there, and doing things, and behaving. Just as people were about to give up on finding the laws, and behaving based on those constituents that would give you a sufficiently empirically adequate theory, maybe it’s premature to give up. Maybe we need to think more generally about what form the laws could take. By 1922, people already realized that it’s possible to have laws that are stochastic and probabilistic. Laws that didn’t tell you in a definite deterministic way what would happen, but simply told you probabilistically what would happen. We knew about these. Einstein certainly knew about this kind of model. One of his great papers in 1905 was on Brownian motion. That helped provide strong evidence for the existence of atoms; it even allowed you to know the size of atoms, and that’s a model that relies on what are called stochastic processes. Stochastic from the Greek word stochastikos, which means aimed or guessed. These are models where the laws predict probabilistic behavior. People knew about these things. Markov had introduced the Markov matrix in the early 1900s, 1906 or something like that. These ideas were in the air, and people were thinking, well, quantum mechanics behaves in an unpredictable way. Maybe the unpredictability can be absorbed into probabilistic laws, but people couldn’t come up with the right set of probabilistic laws at the time.

Over the years, people have tried to return to this question. In fact, in Hugh Everett’s thesis, he talks about an effort made by a physicist named Fritz Bopp who replaced the usual formalism of quantum mechanics with probabilistic laws, where particles move according to probabilistic laws. Everett actually said that this was a promising direction. He didn’t want to study it in his thesis, but this was worth paying attention to. Many other people worked on these proposals, such as Emery Fényes in the 1950s, and most famously, Edward Nelson in the 1960s, but the models were extremely complicated. To get the same predictions that you get from the theory of quantum mechanics, you had to—this is the only political thing I’ll say—gerrymander the electoral districts. You had to gerrymander—I mean this in the same metaphorical sense. You had to fiddle with the laws. You had to start with what we knew was supposed to be predicted and then work backward and design these incredibly complicated stochastic laws in order to get the same predictions. I felt very unmotivated. The spirit was a very interesting spirit. Let’s say particles bounce around or fields, whatever, evolve without Hilbert space, without wave functions. Let’s just give them directly probabilistic rules in a way that makes those probabilities agree with the probabilities we see for measurement outcomes in our experiments. But implementing the idea was hard. And it wasn’t clear that we had the right laws to do it. And the laws that make it work even in the simplest cases require a lot of fine-tuning and fiddling. And when you look, for example, at Nelson’s stochastic mechanics, his laws are extremely complicated. And you actually have to find the wave function first from the Schrödinger equation, and then plug it into these laws, and the laws involve rates that move backward and forward—it’s incredibly complicated. And there were competing approaches. So, Louis de Broglie introduced the pilot-wave theory in the 1920s which was independently discovered by David Bohm. And these theories were originally built on hidden variables, particles, that were guided by the wave function to some extent. When you try to extend these models to beyond non-relativistic particle systems but, like relativistic systems and fields, it gets extremely complicated, and you end up inferring probabilities and stochastic laws again. So there seems to be an understanding that ultimately, to describe most kinds of quantum systems, we’re going to need something like laws that are probabilistic in nature. But none of the probabilistic laws that people knew seemed to work very well. Either they didn’t work, or they needed so much fine-tuning that they seemed unmotivated.

In 2022, I was trying to teach a class, and I got into trouble. We got to a point in the class where I was trying to present a little bit of quantum mechanics, and I was trying to decide how to do it. And I had taught this class many times. Which class? It’s called Introduction to Theoretical Physics. It’s Physics 19 here at Harvard University. And we cover a lot of topics in the foundations of theoretical physics in this class. During the semester, we present some concepts from quantum mechanics, but this comes almost near the end of the semester, and I’m trying to give them a nice self-contained preview of quantum mechanics. But these students are mostly very new to physics. Many of them are just beginning to learn college-level math. I can’t start talking about Hilbert spaces or C star algebras or GNS theory or any of that stuff. I need to find a more user-friendly way to get into quantum mechanics. And every year, I would tear up my syllabus and try again. I was never happy with the way I was doing it. Sometimes I’d say, maybe we’ll start with wave functions. Sometimes I’d say, maybe we should start with matrices. I couldn’t come up with the way I wanted to do it. And in the fall of 2022, I remembered some projects I worked on when I was an undergraduate and even in graduate school where I needed to use the theory of stochastic processes for, you know. As an undergraduate, it was as a special extra homework assignment for a linear algebra class. In my graduate studies, I actually used it in simulations. And, you know, the theory of Markov chains and stochastic processes, they bear some very vague similarities to quantum mechanics. They have probabilistic laws. You represent the probabilities of the system using a vector. The time evolution is represented by matrices. Now, they’re all slightly different. The probabilities don’t involve complex numbers. The matrices are what are called stochastic matrices. But there are such similarities. And so I sat down and said to myself, maybe I can find a way to combine these two theories together. The theory of stochastic processes on the one hand and the theory of quantum mechanics on the other. I had this kind of picture in my mind as if I was walking on the edge of a cliff. On the one hand, you had a kind of classical world, classical stochastic processes, probabilistic processes. And on the other side, you had quantum mechanics. And there was this huge gap between them. And I was looking for a place where the cliffs come closer together and where it’s easier to jump from one to the other.

So I started modifying the mathematics of the theory of stochastic processes and quantum mechanics to try to bring these two pictures closer and closer together. I was doing this without knowing much about this previous work that had been done. The work of Bopp, Fényes, and Nelson. If I had read all those papers, I would have given up because I would have decided it’s impossible. But sometimes you can only do something if you don’t know it’s impossible. I didn’t know it was impossible. So I brought these theories closer and closer together expecting that there would be a moment where I would need to do some kind of jump to go from one theory, classical physics, to the other theory, quantum physics. And the strangest thing happened. The faces of the cliff merged, and there was no longer a chasm. For those who are familiar with second-order phase transitions in statistical mechanics, it was as if I reached a second-order phase transition point. On the one hand, you have the two phases for some materials like water, right? And then you go past a point and the phases merge and there’s only one phase, right? There’s no longer a difference between liquid and gas anymore, right? And I didn’t understand what happened. I was completely bewildered. So I started searching the literature to see if there were other people who had done what I had done, and nobody had done what I had done. I think nobody thought it was worthwhile to try to do this thing that I was doing. But I started reading some of the literature on stochastic processes and Nelson’s work and I was baffled. Why didn’t they succeed when this seemed to succeed? And that was around October of 2022. I know why. I went to the office of one of my colleagues, the great Logan McCarty, who runs science education here at Harvard University. He’s a quantum chemist. He knows quantum mechanics backward and forward. And I went and talked to him and I went to him, and he had a whiteboard, and I started writing down everything I was doing. Because at that point, I realized that the secret sauce, the thing that made this work, is that I had implicitly given up an assumption I didn’t even know was supposed to be made or had been made by anybody. When people model stochastic processes, they usually assume that the process is a Markov process, named after Markov, like Markov, the Russian mathematician who introduced Markov matrices. A Markov process is a process if you want to know what will happen to your system, all you need to know is what’s happening now. And then the laws tell you, at least probabilistically, what will happen next. It’s memoryless. Memoryless, exactly. It’s memoryless. Now, Brownian motion is memoryless. That was Einstein’s stochastic model of atoms. And generalizations of Brownian motion, Wiener processes, and many familiar processes we’ve heard about, random walks, Poisson processes, these are all Markov processes. Nelson’s work, Markov process, they’re all memoryless processes. Now, I had a vague awareness that there’s a subject called non-Markov stochastic processes. And I went and looked in textbooks to see if anybody had done what I was doing. And usually at the end of the textbook, the book says something like, and that’s all there is to the theory of stochastic processes. There is a larger subject called the theory of non-Markov stochastic processes, but this is extremely complicated. And it’s beyond the scope of this book. So it was very hard to find any books on non-Markov processes. And I searched the literature and I discovered that some people were trying to develop tools to deal with non-Markov processes. There are some that can be turned into Markov processes with a change of variables. They become what are called hidden Markov processes. And some, if they’re sufficiently non-Markovian, are very hard to deal with. And so it seems like the Wild West. There’s just this regime beyond the Markov approximation that not many people really felt comfortable exploring. It wasn’t clear how much we needed them. I mean, Markov chains are really good at modeling the world around us. Do we really need to spend a lot of time worrying about non-Markov processes? And what I realized is that I had inadvertently introduced a kind of non-Markov process without realizing it. And as I delved further into this, I discovered that what I was working on wasn’t even a non-Markov process in the most common sense. It was a process of a more general kind that first appeared in the research literature in 2006. And it’s called an indecomposable process. Indec

omposable processes are in some ways the most general kinds of rules, dynamical laws, probabilistic in general, that fail to possess the Markov property. They’re more general than what we usually consider non-Markov processes. I think there’s more than one way to be non-Markovian, and people had a very specific idea of what they meant by non-Markovian. And this is more like a non-Markovian process than it is a non-Markovian process. These processes first appeared when people were trying to understand the kinds of dynamical rules that describe Hilbert space systems, so not what was on my mind. They were already in the Hilbert space picture trying to describe how things in the Hilbert space picture could evolve over time. And for them, the indecomposable process was just a law that could tell you how your process will evolve from one time to another, but it cannot be broken down into smaller times. So you have a law that tells you how to go from zero to final time, but the theory doesn’t give you laws for intermediate times. That’s it. That’s indecomposability. Certainly not laws that can be iterated and composed into a larger law. And if you think about it, this is certainly more general than decomposable laws. I mean, because there’s a larger class where we don’t possess some properties that we might wish to possess, which is decomposability. Newtonian mechanics is decomposable. You evolve a system from an initial time to an intermediate time, and then you can stop, and look at the state of the system now, and then, if you want, you can use Newton’s laws to tell you what will happen next. Nice and decomposable. Maxwell’s equations that describe the evolution of electromagnetic fields are beautifully decomposable. And quantum mechanics, as it’s usually formulated in terms of the Schrödinger equation, is beautifully decomposable. The Schrödinger equation tells you how the quantum state will evolve, and you can evolve it as far as you want, and then read off the quantum state in some way, and then predict where it will go next.

Is there another way to say indecomposability, or sorry, decomposability as a continuous time parameter? That’s a good question. Not necessarily. Because you can imagine an indecomposable process where you’re allowed to go from the initial time to any time you want, so time is continuous. But once you pick one of those times, you won’t have a law that tells you how to go from that time to later times. So time isn’t necessarily discrete here. It’s an excellent question. But what you lack is the ability to break time into smaller time intervals. Yes. So we don’t have much experience with this kind of law until this research paper came out titled Decomposing Quantum Channels by Ignacio Cirac and Michael Wolf. It eventually got published in 2008 in the journal Communications in Mathematical Physics, and people are playing with it. You know, the paper gets a lot of citations. It wasn’t until 2021 that some people introduced the idea of a classical process that could be indecomposable, like just a process where the system has some configuration, it changes, it behaves probabilistically, but the way it behaves probabilistically is indecomposable. That shows up in the figure on page 15 of this nice review article by Mills and Modi, where they mention the idea that it’s possible to have such a process. I didn’t know it existed. I came up with the term indecomposable independently because I didn’t want to use irreducible, which is used in many ways in physics. I didn’t want to use it, I was just looking for a word that would give you, evoke the right idea. And it turns out they use exactly the same terminology. So that was a year before I did it. And they didn’t connect it to quantum mechanics. They were just mentioning, oh, it’s possible, in principle, to have a classical process with probabilistic behavior, where the laws are not rich enough. They are too sparse. They won’t tell you how to go from any time to any other time. You can only evolve from certain times to other times. The process as a whole is not decomposable. What I had inadvertently done is discovered these indecomposable stochastic processes and showed that there’s just a change of mathematical representation that would take such a system with classically looking configurations, it could be a system of particles, it could be a system of fields, whatever we did ‘re trying to model. And if it has indecomposable stochastic laws, then through this mathematical representation, this correspondence, you get the Hilbert space picture. That’s extremely interesting. Yes. And all the weirdness of the Hilbert space picture, superposition of states, interference effects, all of that is associated with what’s called coherence, phases, right? All these things are associated with superposition and interference and coherence. These things can now be understood. They can be given meaning. So, in quantum theory, these things don’t have meanings. They’re just pieces of mathematics that show up in the formalism. We don’t understand what they mean. We know that interference shows up in experiments. We know that these things are called phases or coherence or superpositions, and we see these showing up everywhere. But they don’t have a direct physical meaning. We don’t know what they represent or what they really mean. What I was able to show is that by going from this concrete physical picture of things with definite existence, probabilistic evolution in this indecomposable way, by going from that picture to a picture with the beautiful Schrödinger equation, where the evolution is decomposable, it’s a nice differential equation, and very familiar mathematics, when you go to that picture which looks decomposable, you have to give something up. The indecomposability doesn’t disappear. It becomes all those weird phases and interference effects and coherence. That’s how they show up in this picture. It’s as if there’s no free lunch. If you want a mathematically convenient and simple picture, a picture where you can evolve the system for any amount of time you want, pause, and re-evolve, a nice decomposable dynamical picture, you have to pay a price. And the price is that all this indecomposability becomes these weird phases. In a very loose sense, if you want to think approximately of these indecomposable processes as being like processes with memory, all those interferences and phases, those are the memory encoded in this weird way. And that’s the picture. That’s amazing. Yes. So in this picture, there’s no fundamental rule for the wave function. I mean, you have things bouncing around according to laws that are more general than the laws of Newtonian mechanics or even the laws of a Markov process. And if you want to design a model of measurement, you just design another system that interacts with your first system, and give the whole thing a nice simple set of indecomposable stochastic dynamics. And what you’ll find is that the measuring apparatus will randomly evolve to one of its final readout configurations with exactly the right probability that we expect from Born’s rule. But there’s no wave function. There’s no collapse. It’s just one big stochastic process happening.

So what influences the choice of the time interval to define that time? You said you can choose any time interval, but then you can’t choose a subinterval. Right. That’s an excellent question. So, in order to start an indecomposable stochastic process, you need a time that you can start at, right? And then the process probabilistically tells you where the system will end up to any smooth choice of later time. But is there only one special time that things can start at? The answer is no. What you can show is that when you have a system that interacts with a larger system, so that it’s not left alone, it interacts with a larger system, what you find is that if this larger system exchanges information, with your original system, as would happen if you had a measuring apparatus that reads the configuration of your system, when it does that, you just let the stochastic process tell you what happens, right? There’s no postulate here. I let it evolve. And what you discover is that you get a splitting event. The evolution splits at that moment, from the point of view of the smaller system. And now there’s a new time where the evolution is broken, and you can now start again from that new time. So, in the indecomposable stochastic process, it doesn’t mean there are no times where you can decompose. There are times where you can decompose the evolution, but the times are not given to you initially. They’re not fundamental. They arise from the interactions between systems. Now, when you ask how this looks from the Hilbert space point of view, in the standard picture of quantum mechanics, from the Hilbert space point of view, how do those splitting events look? They look exactly like decoherence. So decoherence, again, is when a large system interacts with a smaller system, the environment comes in, interacts with the system, exchanges information, and we get these changes in the quantum state of the other system. Changes that don’t define a single answer, but change it in some way. These changes wipe out the coherence effects. They kill the interference effects. From the point of view of the indecomposable stochastic process, they look like the generation of a new moment, a new splitting event, an event where the evolution can start again. And this happens at the end of measurements, and this is a physical process in this picture, and it coincides with what we might call measurement. But it’s generated by the laws from this picture themselves. Okay, I’m bewildered. So, splitting events, are they just another renaming of what measurement is? Good question. So, in a way, if you give me a second system that I design explicitly. So let me phrase it differently. In the textbook postulates of quantum theory, measuring apparatuses are treated as being outside the formalism. They’re kind of in the background. They’re kind of like the person behind the curtain, right? We perform the measurement, and we just change the quantum state of the system being measured, but we don’t usually put the measuring apparatus explicitly into the description itself. If you try to do that, if you try to put the measuring apparatus into the description, what you find after the measurement is that the measuring apparatus and its possible readout configurations become entangled with the possible configurations of the system being measured. There’s this entanglement that happens. And the question is, well, then what? Now they’re entangled, what do I do? What you’re supposed to do is collapse to one of those entangled branches. That’s the collapse. Now, if there’s a larger environment that comes in, and the larger environment looks at the measuring apparatus, it becomes entangled with the measuring apparatus. And there’s this process, this kind of formal process where you can kind of ignore the environment. And you see that there’s a suppression of all the interference effects, but there’s no yet determination of a single outcome. You have to at some point pull out the fifth postulate and activate it and collapse to one thing at some point. In this indecomposable picture, you can bring in this measuring apparatus, you can bring it in, and you let the overall system evolve stochastically. And what you find is that, by including it in the environment, the environment comes in, and you can treat them all as actual systems. None of them are outside the formalism, they’re all designed as systems within the formalism. And what you find is that when you let these systems evolve, the system being measured and the measuring apparatus evolve most likely toward the configurations that we expect from the collapse postulate. And because of the environment, you can show that measuring apparatuses in the objective system, undergo a splitting event at the same moment of interaction that we call measurement. And this splitting event means that you can now restart the laws and start applying them from this moment. This splitting event is not postulated, it’s not a postulate. It’s something that just emerges from the usual rules of probability theory, when you actually model the systems explicitly. So, it’s a measurement in a way, but it doesn’t require a conscious being, it doesn’t require a human being, any system that has a very large number of moving parts, we call them degrees of freedom, in contact with a nice large environment with a large number of degrees of freedom, and it will work to a very high degree of accuracy, and it will produce measurement outcomes that probabilistically agree with what we expect from textbook quantum theory, and it will generate a moment where the stochastic dynamics of the universe. The system being measured in the measuring apparatus can be restarted and have its own dynamics. So yes, it’s a measurement in a way, but it allows us to get inside the measurement. It lets us see the measurement actually happening, and it gives us a way to understand why it happens

Measurements with this thing, instead of just assuming that they lead to the collapse of wave functions. Do you need to mention three different systems? Then the environment, the apparatus, and the measurand? Yes, that's right. And we demand that in standard quantum measurement theory anyway, in the usual textbook theory. And the environment as well? The environment is required, yes. Yes. Because the usual argument is that the environment is what generates the decoherence between the measuring apparatus and the system being measured. But when you say the environment, I mean that in any realistic measuring apparatus, you have, you know, a part of the measuring apparatus that is actually directly involved in the measurement, but then you have the rest of the measuring apparatus. I mean, it's a big body, it's sitting in the box, and it has dials, and it has other experimental things. I'm sorry, that means I will not be a good experimentalist again. But this, yes, there is always an environment approximately.

So, you first have to describe the environment, the apparatus, and the measurand classically, then you do some version of the quantum measurement procedure? Yes, so you can design them classically, in the sense that they have classically-like configurations. The measuring apparatus is some system, or physical body that can be in different possible configurations. Among those that we list at some level of coarse graining, we list, you know, the system is empty, it doesn't show a reading yet, the system shows an up spin on the dial, the system shows a down spin on the dial, you know, but these are concrete configurations made of certain configurations of constituent atoms, whatever you want. And that means their configurations, their ontology, their physical existence is conceived of in a pretty classical way. The quantum part is that the laws that they obey are not deterministic differential equations like Newton's laws, or Maxwell's laws of electromagnetism. And the laws that we give them are these non-separable stochastic laws, which we don't typically see in classical systems. The claim is that when you give these systems these more general kinds of laws, that's sufficient to make them behave like quantum systems.

So what's stochastic or random here? Good question. So all these systems have configurations, and these configurations evolve according to stochastic laws. That means that the configurations change over time in a probabilistic way. And now, why do they change in a probabilistic way? I don't know. These are axioms, right? At some point, you have a physical theory, you have to propose some axioms. I mean there's a position in how we think about foundational questions in philosophy called foundationalism. You have to start somewhere. The standard textbook version of quantum theory, which works extremely well, and I certainly don't claim that any of the predictions it makes are wrong. They're all right on the mark. These axioms are axioms, and they are very peculiar. We posit Hilbert spaces and state vectors or density matrices and the Schrödinger equation and unitary operators and all the collapse postulates, and we posit them because using these axioms we get the right predictions empirically. This picture I present also has axioms. The axioms are simpler. Each system has a set of possible configurations, classical configurations, and the laws are these non-separable stochastic laws. That's it. Now, if you want to ask why the laws are non-separable stochastic, I don't know. The best I can say is that when we try to explain things, there's an argument that you want to start with the least restrictive probabilities, the most general probabilities, and then argue that under certain circumstances you get more restrictive probabilities. Non-separable stochastic laws are the most general kind of laws you can imagine. Maybe you can go beyond that. I don't want to say they're definitely the most general, but they're certainly more general than Markov laws. They're more general than the usual way we think about non-Markovian laws. They're more general than deterministic laws, laws of the differential equation kind. To me, it makes sense that you would start with the most general kind of law, and then explain why in certain special circumstances we expect to see Markov processes, and why in certain circumstances we expect to see deterministic second order differential equation kind of laws. That's the task of understanding the classical-quantum transition, understanding the so-called classical limit. How do we go from the laws of quantum mechanics to laws that look like the laws of Newtonian mechanics or look like classical Markov processes? Those arguments also proceed in this picture. We don't start with deterministic laws. We don't start with Markov chains or anything like that. We start with these more general laws, and then we adapt the usual arguments that people usually use in trying to do this to arrive at the classical limit, the classical regime, we see why under certain circumstances we get large systems whose behavior looks classical, whose laws look deterministic, or at least look like probabilistic Markov laws. To me, that's the right direction for interpretation. You start with the most general, least restrictive kinds of laws, and then you move to more restrictive laws. I think I'll say that I don't feel the need to explain why particles or things behave probabilistically. Probabilistic behavior is the default behavior. That's what happens if we don't have a good reason to believe that the laws should be deterministic. And then the goal is to explain why under some circumstances we get deterministic predictable behavior. But the truth is that I don't have a deeper explanation for why we expect particles to jiggle probabilistically. That's what it means to have a non-separable stochastic law or a stochastic law in general. And your probabilities, are they classical probabilities or are they quantum amplitudes? That's a great question. They're classical probabilities, and that's kind of key.

So this basic picture, the claim that every quantum system at its beating heart is really a system with configurations. If it's a particle, we're talking about where the particle can be. If it's a field, we're talking about the different intensities the field can have in space. If it's a system of abstract bits, like in a computer, it's a configuration of zeros and ones. That's what I mean when I say classical-like ontology or classical-like configuration space. And the probabilistic laws assign ordinary probabilities to these things. They say that the system starts in some configuration, and then the non-separable stochastic law takes the form of laws that tell you the probabilities of where it will end up. And these are just ordinary probabilities. They are real numbers between zero and one. They sum to one. There are no complex numbers whatsoever in this picture at this stage. But if you want to describe the same process in a mathematically more convenient way, I mean, non-separable processes are actually quite inconvenient to work with, especially if your system is complex, because we don't have this separability of laws. It's very difficult to use directly. So instead, we use this mathematical change of representation into a Hilbert space picture. We translate all the ingredients in this underlying process into a Hilbert space picture. In the Hilbert space picture, we have nice differential equation kind of laws, the Schrödinger equation. We have complex numbers appearing. And then the probabilities that you have in the non-separable stochastic theory become what we call the squared moduli of the complex numbers. The complex numbers whose squared modulus you take to get the probabilities are called amplitudes. So only when you go to the Hilbert space picture do we need amplitudes. And amplitudes do all sorts of funny things. They interfere with each other. If you want to write down a process where the system can go one way or another, you're supposed to consider all the possibilities and then add all the amplitudes together, and there's interference, right? But all of that happens in this kind of mathematical representation, this Hilbert space side of things. And the claim is that this doesn't tell you what's actually happening in the system. This is just extremely convenient mathematics. And again, it produces all the right predictions, right? The idea here is that you recover the usual axioms of quantum theory, but now you understand where they came from. And now they're no longer inconsistent anymore, because at the fundamental level, there is no special role for observers. There is no special role for measurements. All the processes we think about in quantum mechanics are just ordinary processes. They just look weird and strange when you see them on the Hilbert space side.

Have you encountered any resistance from people who like their quantum mechanics mysterious? Yes. Explain. Well, I mean, quantum mechanics is very exciting. Actually, let me rephrase that. That doesn't mean that they like their quantum mechanics mystical. Have you encountered any resistance from people who like their quantum mechanics mysterious? Puzzling. Yes, they like it to be shaken not stirred, right? I mean a little bit. Yes, I think there's a sense that this picture is deflationary. It deflates. It takes away peculiar pronouncements. Oh, the cat is alive and dead. And deflates them and says, actually, it's really one or the other. And we just represent it mathematically by a state that looks like a superposition of alive and dead because it gives us a separable evolution we can use. It's easier to study mathematically, but that's not reality. It deflates that. Yes. Just to clarify, deflationary is a philosopher's term for saying there's nothing to be seen here, folks. Just calm down. Yes, it's basically like sucking the air out of a balloon, right? It brings it down to earth, and that's basically what this does. I think it makes quantum mechanics, in some ways, less exciting. And you know, while I was working on this, I kind of felt that when young people enter physics and they start learning about quantum mechanics, it's incredibly exciting to think that you might be able to understand what happens when Schrödinger's cat is alive and dead. And you know, particles can go through both slits in the double-slit experiment at the same time and interfere with themselves. And you know, everything is entangled with everything else. And maybe when we think about things, we humans are special. We collapse things. That's boring quantum mechanics. It says that at some level, quantum systems, if things are done right, might just be kind of banal. They're just there's no Hilbert space wave function. There are just things, atoms, chairs, whatever. And they just evolve in a non-deterministic way, a highly non-deterministic way. Okay, still probabilistic. We can still assign probabilities to things because you can imagine laws that are so non-deterministic that we can't even assign probabilities to them. But we can assign probabilities to them. But then a lot of the magic is gone, right? There's no special rule for humans. In a sense. Means what? The history of science has been pushing humans further and further away from the center of the story. You know, at first we were the center of the universe. Then we weren't. Then maybe we were the center of all observation and quantum mechanics. And this takes us out of that completely. There's nothing particularly special about human observation or the human mind. And I guess, with evolution, we might not be soon the smartest people on this planet, the smartest beings or entities on this planet. Um, so I don't know. I mean, I can only speak for myself. I felt a little sad when I thought there's a way of thinking about quantum mechanics where there is no wave function. Nothing branches into multiple realities.

So you mean there's no fundamental emergent wave function? Well, good. It emerged. Yes. So wave functions play their role, so there's a way to take Newtonian physics and put in kinetic energy, which is energy associated with things moving and potential energy, which is energy associated with the configurations of things. And you can take these two quantities. They're more abstract than being like particles and forces and things, but you can imagine kinetic energy and potential energy. And you subtract them from each other, which is kind of weird. Usually we think total energy is the sum of these things. You subtract them and then you can integrate this thing over time. And you get a mathematical construct called the action, the action functional. We actually cover this in my class right now. The thing you integrate kinetic energy minus potential energy is called the Lagrangian. At least in the simplest cases, it's kinetic minus potential energy. And you can use this to encode the laws of your system. You can use this to describe how the system behaves. But you can ask what is the action? Like, is it a physical object? You can make the action evolve in time. You get what's called the principal Hamilton function. It obeys a partial differential equation. And I didn't mention it, but this is the beginning of Hamilton-Jacobi theory. And by studying the action evolving in time, the principal Hamilton function, and its differential equation, Schrödinger was led to discover the Schrödinger equation. It was a beautiful extrapolation of the partial differential equation satisfied by this action. So this action is a function of time or principle, the principal Hamilton function is very much like the wave function in quantum mechanics. So much so, as I said, that it inspired Schrödinger to develop the wave function. In a sense, Schrödinger's wave function was an exponential of the principal Hamilton function. And now you can ask, what's the meaning of the principal Hamilton function? Is there a physical object in the world that evolves according to this partial differential equation, the Hamilton-Jacobi equation, the equation satisfied by this principal function, this integrated kinetic minus potential energy? I think most physicists would say no. They would say it's just a mathematical convenience. It's an auxiliary to the theory. We can use it to generate predictions. We can use it to predict. And that's great. It's beautiful mathematics. But there's nothing in physical reality that it describes. And what I'm saying is that the wave function is basically like that. It belongs to the same conceptual or metaphysical category as Hamilton's principle works. Sure, it's in the math. It's convenient. But there is nothing physical that it describes. And because there's nothing physical, there's nothing branching. There's nothing that splits into branches. So you would never think about the possibility of multiple worlds or anything like that. There's nothing in the ontology of the picture as I've presented it that would play the role of branching realities. So things like many-worlds don't even get off the ground in this picture. So I think that's what I mean when I say there's no wave function. I mean it in this fairly precise sense. I don't mean that we shouldn't use wave functions in the mathematics or use them to predict. But we should be a little more humble in the way we talk about them.

Have you talked to Sean Carroll about this? That's an interesting question. Not yet. But at some point we will hopefully talk. Yes. It's an interesting question because I haven't traveled much in the last year or so. So our paths haven't really crossed. But I would like to have a conversation about this topic sometime. Any Everettians or many-worlds? People? A little bit. A little bit. But I think you mentioned mysticism. You talked about how people like their physics. The truth is that people get good at doing physics a certain way. And we develop intuitions. When you learn to play a musical instrument, you learn to play the piano, you learn to play chess, you learn gourmet cooking. And you do this over many years. And if you have the right mix of motivation and talent and skill and the right environment and the right mentors, the right mix of all those factors, you will end up becoming very skilled at it. And you can do amazing things. You can play beautiful musical pieces. You can prepare a fantastic cuisine. You can become a grandmaster in chess. And if someone comes along and says, I think you should do it differently. You should go back to the starting board. You should sit on the piano bench like this. And you should hold your hands like this. Or you should play chess. And you should start the pieces in a different position. And we should change all the rules. The chess pieces should move differently. You're going to put the person back to square one. And you can imagine why someone might not want to do that. And not for any negative reason. It's just that they've honed themselves to do something really well. And asking them to go back to the drawing board and start all over again is a big ask. And so I would say basically that anyone who's been working in quantum mechanics a certain way for their entire career, who has developed all sorts of intuitions, they have a good feel for what to do with wave functions and things. You know, I can imagine someone, in this position, being hesitant to go back and rethink all the foundations from scratch. Especially if what they've heard for most of their careers is that the foundations are fine. Quantum mechanics is just a tool for generating predictions. Why should I go back and think about any of these things? So I don't think I'm still I don't criticize any of this position. But what I certainly found is that newer people in physics, students, tend to be more interested in talking about a project like this. But there are absolute exceptions among professors as well. I mean, I have, you know, some wonderful colleagues who have been incredibly receptive and open, and we've talked about this project a lot. And I've gotten really a lot of good input from people. So, you know, but I think it's a mixture of intuition building and also what draws us to study quantum mechanics. I mean, quantum mechanics is a strange theory. I mean, the belief that you're going to spend your time imagining the world is a wave function or a state vector or an abstract object living in a high-dimensional configuration space or in the somewhat more modern sense in a high-dimensional Hilbert space. Hilbert spaces are beautiful. The mathematics is incredibly elegant and has a certain kind of spice to it. But you get complex numbers floating around. We don't know why, you know, at least in the way we usually talk about it. I can imagine being romantic about that. I was. I feel that's what drew me to quantum mechanics. So I can certainly see why some people would want to hold on to that. And I think of all the people who take a romantic view of quantum mechanics, I don't think anyone takes a more romantic view than Everettians, the people who work on the many-worlds interpretation because their picture of the world is not only strange. It's profound, right? The idea that every time a quantum event happens, the universe splits into two or three branches, but in a more sophisticated and more accurate version of Everett's approach, it's an infinite profusion of branches. I mean, it's mind-blowing and stunning and forces you to confront some very profound questions in metaphysics, in philosophy of mind. It's just incredibly rich with inquiry. I can imagine that people who work in that area would want to hold on to it. So I'll give you two examples because I can relate to that, being stubborn and opinionated. So when I was learning about the classification of quantum mechanics and reading papers like, okay, here's another way you can think about quantum theory as a dagger-compact symmetric category. And I look at it and I think, okay, but so what? I still have that, so what? But I think part of it is that I'm used to doing quantum mechanics in an alternative way. Now, my father sets up his office in such a way that if you move one piece of paper, he will be upset with you, even if it's completely out of the way, he keeps everything in his mind. I had to install Windows XP two years ago, Windows XP on his computer because he's familiar with Windows XP and wants to use DOS and can't run QBasic on Windows 10 without some emulator or virtual machine. So my father is like, what you classify as, or what you describe as some people who are used to quantum mechanics the old way, at least when it comes to only not hating it for being deflationary, but hating it for being different. Now the question could be, well Jacob, what does this give me? What does your approach give? Why should I learn it? You just said it looks like ordinary quantum mechanics. Okay, great. Maybe for you it's a more beautiful version, and maybe for someone else it's more complicated, but whatever, what does it give you? What's the payoff? That's a great question. So the first payoff is a correction of the axioms. The axioms had this mystery or inconsistency or incompleteness to them. They had this puzzle about what measurement is that seems to play a role in the axioms, but is not defined by the axioms. And you'll encounter problematic thought experiments like the Wigner's friend experiment. And the Wigner's friend thought experiment is that it's not the same as the black hole information loss problem by any means, but there are some similarities. There's this big problem in quantum gravity. Black holes seem to radiate according to work going back to Stephen Hawking. And there's this tension between black hole evaporation quantum mechanics and unitary evolution or Schrödinger evolution, which is information preserving in some sense in quantum mechanics. And the fact that black holes are supposed to not be able to let information escape. There's this tension between all these things. That's the kind of tension that we see in thought experiments like the Wigner's friend thought experiment. And it's not as if the black hole evaporation problem is easily accessible experimentally. I mean, to go out and do experiments on black hole evaporation, you would have to prepare 10,000 identically prepared black holes, which we don't have to do. Wait for many times the age of the universe for them all to evaporate, and make sure that they all evaporate inside a perfectly sealed room with perfect detection mechanisms to collect all the Hawking radiation that comes out with sufficient sensitivity that we can detect, you know, incredibly tiny entanglement effects, right? Then we subject all the data to a level of analysis that we think might be computationally impractical. I mean, this is not a practical problem, at least as far as we know so far that could change. But as far as we know, it's not a practical problem. Nevertheless, this is a serious area of research. Whereas inconsistencies in quantum mechanics are not considered serious for some reason. I don't quite understand. There's some sociology there. But what I would say to quantum physicists is that one advantage of this approach is that it resolves some of those inconsistencies, which is maybe in itself a good thing. It does so in a way that doesn't require us to model only one kind of quantum system. One of the drawbacks of pilot-wave theory like Bohmian mechanics is that it works really well with very simple kinds of systems that consist of a finite number of non-relativistic particles, but it has resisted precise generalization beyond that. So it's nice to have a more independent and simpler model than the Bohmian approach. And it doesn't face many of the puzzling metaphysical riddles that one has to grapple with with Everettian quantum mechanics. You know, we're not making millions of copies of ourselves in this picture. So it provides, I think, one way and maybe at this point, maybe the only appealing way, in my opinion, to resolve these inconsistencies with quantum mechanics, although mileage may vary and I don't want to speak for everyone by any means stretches. That to me is already something. And the second thing is that it provides a picture. Quantum mechanics, when I teach quantum mechanics to students, they expect me to talk about waves propagating in three-dimensional space. And when I show them the axioms, many of them are quite disappointed. They're like, but where's the picture? Isn't it like there's an electron and it emits a photon? And I say, well, the textbook axioms don't really talk about an electron here emitting a photon there. According to the axioms, that's just added for color. We draw pictures of electrons emitting photons only to help tell the story. But none of that is actually sanctioned by the axioms of quantum mechanics. There's also a problem that there are many phenomena that occur in the world that don't seem to be measurements. You know, when early gases mix in the early universe, when distant stars collide, when a piece of aluminum is sitting on the surface of Mars and doesn't collapse under its own weight, when birds forage, when humans fall in love. These don't seem to be measurements. Yet the quantum axioms the theory only makes predictions about measurement outcomes, probabilities of measurement outcomes, averages of measurement outcomes weighted by probabilities of measurement outcomes. And that's it. That's a very narrow class of phenomena. How do we go from that narrow class of phenomena to the broader class of all the other things we think happen in nature? Or are we just going to say that our talk about all those other things, about birds foraging and early gas mixing, is just for color, just storytelling? And this allows quantum mechanics to be a theory of a much larger class of phenomena, not just a narrow class of measurements. I call this the class problem in textbook quantum theory. But ultimately, I think physicists will be interested in applications, practical applications. So—or you were hoping. Well, we were hoping, but yes. But in my experience, physicists, you know, want to know, what can we

What should we do with this? I think there are a few things you can do with this. The first is that irreducible random processes, at least classical irreducible random processes, are currently three years old. This is a completely unexplored area of dynamical systems theory. It’s like opening a full sketchbook and turning the page and finding a completely blank new page. How often does that happen in mathematics or science, that you turn the page and you open the door to a completely new area of research? I don’t know much about the mathematics of irreducible random processes because nobody knows it yet. I think that’s exciting. I think it would be very exciting for people to think about these kinds of systems and discover their properties.

Perhaps the best way to study an irreducible random system is to write down its Hilbert space and essentially do quantum mechanics. Just as sometimes the easiest way to study a Newtonian system is to write down its Lagrangian and do Lagrangian mechanics. Maybe we can understand things about these systems according to their own advantages. Maybe when you don’t start from a Hilbert space, but you start with these different axioms, you can point to new ways to generalize quantum mechanics that were impossible to imagine before. If you start with a Hilbert space, that limits your imagination about what you can do as well. If you don’t start with a Hilbert space, you’ll start with a more general kind of probabilistic system as I’ve been talking about in this picture, this irreducible random process. Maybe there are deformations you can make to the assumptions, changes you can make to generalize these systems in ways that don’t give you a Hilbert space. You can generalize these systems and get something else that doesn’t give you a Hilbert space that you could never have accessed if you had started from Hilbert spaces. And maybe we need such a generalization in order to make progress on fundamental questions related to quantum gravity. So I think that’s another potential area of application.

And there’s another area, which is that if there’s this correspondence between Hilbert space representations of quantum systems and classically appearing probabilistic systems, especially beyond the Markov approximation, this might suggest efficient ways to simulate classical non-Markovian random systems using quantum computers. So one of the open questions is if we had a quantum computer today, what would we use it for? There are already a number of known algorithms. There’s no proof that the algorithms are more powerful than what you can do classically, but it’s widely believed that many things you can do using these quantum algorithms are much more efficient than what you can do classically. But it’s not as if every question you want to do on a classical computer can be done more efficiently or more quickly on a quantum computer. So knowing what kinds of things a quantum computer will help us do more efficiently is a really open question. And when you establish a correspondence between quantum systems and a large class of random systems, you open the door that there are some problems that might be difficult to simulate classically that are easier to simulate quantum mechanically. I don’t know exactly what those are yet. And that’s an area of research that I’m working on right now. But I’m talking to people who are working in quantum simulation theory of classical random processes, and we’re trying to determine maybe there are ways to use these correspondences to provide more efficient ways to simulate these systems and provide a new application for quantum computers.

Now, there’s also the possibility that under some circumstances, irreducible random systems might behave differently than quantum systems, at least textbook quantum systems. There might be slightly different predictions regarding the Wigner’s friend thought experiment. Unfortunately, doing a Wigner’s friend thought experiment is probably beyond our experimental capabilities for the next thousand years. Who knows? But there are some cases where you want to model the quantum system, and you need to make additional assumptions in order to make actual predictions, assumptions that go beyond the textbook axioms. For instance, if you want to design a very complicated quantum system, you might have to add some extra axioms about, you know, what are the typical initial states of the system, and what’s the typical kind of evolution that we expect. These go beyond the textbook axioms. And it’s possible that when you start with different axioms, you’re actually modifying some of those assumptions. And therefore for some classes of very complicated quantum system, it’s possible that you get slightly different predictions. But these are all highly speculative claims. There’s a lot of work that needs to be done to determine whether any of these things can work and when. So I want to be very clear, we’re just at the beginning here. But even the existence of these correspondences at all, I think opens up a lot of doors. And there’s nothing more exciting to me in science or philosophy than having doors open. Okay, this is definitely cutting edge, like the latest developed in the last two years. And also, it may be widely applicable soon and may already be on the right track. Does it offer any insight into QFT, which relies on Fock space, which relies on Hilbert spaces? Yes, so that’s an excellent question.

So quantum field theories fall within the axioms of textbook quantum mechanics. Those axioms I mentioned, and Hilbert spaces and all that stuff. A quantum field is a particular kind of system that lives within the framework set by those axioms. What makes quantum fields, well, there are many things that make them complicated. The one thing that makes them complicated is that they contain infinitely many moving parts. Unlike a system that consists of five particles, they have infinitely many moving parts. We see that these systems have infinitely many degrees of freedom. And that causes all kinds of mathematical complications. When working with quantum field theory, like the way ordinary physicists use quantum field theories today, you often have to regularize the theory. I mentioned Planck introducing Planck’s constant for regularization. Yes. So the regularizations that we use temporarily make many of these infinities finite. So that we can do the calculation. And then what we do is we try to send the regulator to infinity. And usually we can do that. Then it goes away. And we think we have a good, robust prediction that doesn’t depend on the regulator. Unlike Planck’s constant, the regulator goes away and we think we have a robust result that holds in the infinite connected limit. And there’s a beautiful and complicated story called renormalization and effective field theory that’s all about how to do that in a mathematically consistent way, and in a reasonably consistent way, and generate good predictions. I won’t have time to go into all of that. But ultimately, it’s true, this is still just a quantum system, a very complicated quantum system. And insofar as you can generate axioms for any quantum system, you can generate them for quantum fields starting from an irreducible random process indicator. According to this picture, a quantum field is a collection of non-localized entities, spread out in space in configurations. And these configurations evolve in this irreducible random way that can be modeled in the language of Hilbert space. In a sense, it’s like any other kind of system, you’ll still face the same mathematical complications, and maybe new complications because of the fact that we have infinitely many complications and you have to do mathematical tricks to make these things finite. But yes, the answer is that quantum fields fit into this framework. But it will inherently be very complicated because quantum fields are extremely complicated.

And does it reproduce the precise assumptions of quantum mechanics? Or are there some deviations such that there is an experimental way to test whether the random approach is the more correct approach? That’s a good question. So at the level of correspondence, there’s just a map that takes you from the components of the irreducible random theory to the corresponding Hilbert space description of the quantum Hilbert space representation. There are principle deviations because collapse doesn’t fundamentally happen in this picture. However, the deviations will be roughly similar to any no-collapse formulation of quantum mechanics. So Bohmian mechanics, de Broglie-Bohm pilot-wave theory, where the particles are guided around by wave functions, ready always in quantum mechanics, you know, many-worlds interpretation, these are all non-collapse interpretations. Collapse doesn’t fundamentally happen. There’s no axiom of collapse. And so you will get small discrepancies in the predictions of these theories compared to textbook quantum theory. And I can make this very explicit. Go back to Wigner’s friend. According to textbook quantum theory, Wigner’s friend inside performs a measurement which leads to a collapse of the quantum system inside the box. That system has collapsed. It’s collapsed for everybody. It’s collapsed for Wigner’s friend in the box. It’s collapsed for Wigner outside the box. It’s just collapsed. In a no-collapse approach, like many-worlds or Bohmian mechanics or my approach, there’s no actual collapse that has happened. And what that means is that the wave function that Wigner uses from the outside, the quantum state is an uncollapsed quantum state. It contains superposed position interference phase effects that have not been erased. They haven’t been erased because there wasn’t a collapse. So in principle, Wigner from the outside, using a sufficiently powerful measuring device, could detect those small discrepancies, which might be inconsistent with textbook quantum mechanics. But detecting them would be extremely difficult. So there’s a paper from a few years ago, Scott Aaronson and Lenny Susskind and I and another author where we showed what people call quantum necromancy, which just means that if Wigner could design a sufficiently sophisticated and sensitive experimental apparatus to detect those small discrepancies between a no-collapse interpretation and textbook quantum theory, that apparatus with some relatively minor modifications could take a long-dead cat and bring it back to life, which we usually consider a thermodynamically impossible feat, right? That’s what they call necromancy, like magically bringing the dead back to life. It would be more like unscrambling an egg, right? Like taking a scrambled egg and returning it to the original egg. I mean, this really goes beyond any technological capability that we can imagine. So yes, it does give slightly different predictions, but only to the extent that other no-collapse interpretations give the same slightly discrepant predictions. And unfortunately, these discrepant predictions are so tiny that they’re extremely difficult to measure. Maybe measuring them is easier than measuring the black hole information problem. But kind of on the same level of extremely tiny suppressed effects.

Okay, before we end, I want to know what is a particle from your perspective? And also is your view called an interpretation? Is it a theory? Is it a correspondence? What do you call it? Why don’t we just settle that now? I don’t have a good answer to that question. I think you can call it whatever you want. One can think of it as a physical theory. It has variables and components in it, and those components have laws. So it’s a physical theory. It’s a physical theory that makes the same predictions that quantum theory has made in all realistic scenarios to my knowledge. You can think of it as an interpretation of quantum theory in the sense that if you start with the formalism of quantum mechanics and ask, how do we interpret things? What do things mean? This provides a platform to actually talk about what’s really there in nature. I should say that it’s not a one-to-one picture. A given quantum system might be represented by many different irreducible random processes just as a Lagrangian, or better yet, a Hamiltonian formulation of classical physics can describe many different Newtonian systems. There’s no one-to-one relationship between these pictures. You can think of it as a formalism because it’s a new mathematical framework. I think what you can’t call it is a recipe or prescription because it’s more than just a list of steps to follow. It’s more than just an algorithm, just as the axioms of quantum theory are more than just an algorithm to follow. It actually tells a story of the things that happen. I don’t know if that answers your question. Do you want to follow up on that or do you want me to talk about particles? What is a particle? Classically, of course, a particle is a point-like entity characterized by a mass of some kind. The mass could be zero. I think it’s a tachyon. It could have some weird imaginary mass. It’s characterized by spatial degrees of freedom, meaning it has a location in some conception of physical space, usually x, y, z coordinates when I think about it that way. It will generally have different locations, at different times, which we can think of as like a trajectory. If it’s truly a point-like particle, it has no internal structure. It has no size. However, it can carry certain other quantities. It can have other attributes like electric charge, which is related to how it interacts with fields and other kinds of particles. It can have an intrinsic form of angular momentum. You can have kinetic energy, but kinetic energy can kind of freeze out and become a kind of rest energy, like the internal energy of the particle. Angular momentum is a kind of orbital momentum, it’s a kind of rotating momentum, but you can also have frozen orbital momentum. And that’s called intrinsic spin. Particles can carry this intrinsic spin which can be converted into other forms of angular momentum, just as rest mass can be converted into other forms of energy. That’s what a particle is. It’s a thing with this collection of possible properties.

In quantum mechanics, at least since Wigner’s work on representation theory, particles have been understood as the simplest kinds of quantum systems that have well-defined nature or behavior under spacetime transformations. If you asked what’s the simplest kind of quantum system whose Hilbert space is as small as possible, it doesn’t involve anything weird, but when we move in space or we rotate or we change inertial reference frame, we change frame of motion, we just get a new element of the same Hilbert space. We don’t get something new, but it’s minimal. It’s irreducible. Then you have a problem in mathematics. What are the irreducible unitary representations, which just means they’re represented by certain kinds of operators in Hilbert space, representation of the spacetime symmetry group, the Poincaré group, the abstract representation of moving in space, changing frame of motion, rotating. This is a problem in mathematics. You can classify them, and what you find is that there are distinct kinds of Hilbert spaces with these properties, characterized by mass and spin and charges. They have kinematic degrees of freedom and so on. This is usually the way we think about particles in quantum mechanics. This is all we can say in quantum mechanics because all we have is Hilbert spaces in the textbook formulation. If you’re going to describe a system of particles, you have a Hilbert space. All we can do is talk about what the Hilbert space is, what its structure is, and the structure of the particle is related to these spacetime transformations. In the picture I’m proposing, particles return to the classical picture. A particle is a point-like object in space, in three-dimensional physical space if we believe that physical space is three-dimensional. With different properties or dispositions that interact with other particles. Dispositions? Yes. This means that when you interact with a particle, it might be disposed to give you certain outcomes on your measuring apparatus, that’s all I mean. If you measure its position, you’ll read out what its position is, but if you measure some more complicated features, what you get on your measuring apparatus will be a more complicated outcome of interacting with the particle. It has a disposition to produce certain kinds of behavior. In metaphysical talk, we can say that some properties are categorical. They’re explicitly there as they are, and others are dispositional. Meaning you look at it and what you see is actually like how the particle is disposed to show you what it will do. But ultimately, particles are very much what they are in the classical picture. Just point-like entities with certain properties, different locations through time. So that’s all a particle is according to this picture. And I should say that I’m not saying that particles are the fundamental ontology of nature. I don’t know what the fundamental ontology is. But so far in your theory, what is the fundamental ontology? We go back to the old classical way of doing things, which is that if you want to model a certain system, you propose a certain ontology for it. You have to say what you think the configurations are. If you want to model particles, that would be arrangements of particles. If you want to model fields, it would be non-local field intensities distributed across space. If you want to model strings, if you want to model bits on a computer register, you choose the right ontology for the right kind of system and then you model it. It’s possible that there is some truly fundamental ontology of the whole of nature. I don’t know what that is. I think for some people, especially people working in Everettian quantum theory, the idea that the fundamental ontology is state vectors or wave functions in a Hilbert space is a very appealing idea. That means we already know what the fundamental underpinnings of all of reality are. Hilbert space, there’s some Hilbert space and there’s an object or objects in the Hilbert space and that’s the fundamental ontology of everything. I’m not saying that. Hilbert spaces are not fundamental in this picture. We go back to needing to be somewhat metaphysically modest. We know about certain systems. Some systems look like particles, some look like fields. Maybe particles emerge from fields and we don’t need particles as a separate category. Maybe fields are fundamental, maybe they’re not. I think it’s premature to try to guess what the fundamental ontology is right now, and in this picture you don’t have to. All you have to do is choose the ontology that you need for the model you want to talk about, particles, fields, registers on a computer, and then proceed from there. Maybe someday we will discover that there’s one ontology with one existence from which we can derive all the other models. That would be a very exciting day, but I don’t think we can guess what that is today and I don’t think we can just say it’s Hilbert spaces and be done with it. I think that’s premature.

There’s a lot more I want to talk to you about. And we haven’t talked about causality either. There’s a whole other connection to causality. Reichenbach’s common cause. Right, we haven’t talked about Reichenbach’s common cause principle. And that’s another reason why I think this work is likely to be exciting because it has some connections to causality that I’m very excited about. I think we should explore that next time. Yes, perhaps we should talk again. Professor, it was wonderful seeing you in person. I have been following you, again, I’ve been following you politely. Diplomatically, whatever. You understand. For some time now. Why don’t we end, I know you’re about to head off to see your students, why don’t we end with your advice for the students? Oh. Actually, two. Your advice for the students watching and the researchers watching. I’ll just use my fallback advice, which I think has been really useful for me. I’ve been told this by people and maybe I’ll share it. We all come to the work that we do with a distinct set of strengths and weaknesses, just as I’ve just said. You know, each person is a certain set of strengths and weaknesses we’re like a point in a very high-dimensional configuration space or parameter space. So we’re not ranked. You can’t rank people. We’re not a ranked set. It’s very easy for students who are studying science to think that they are operating under a linear ranking system. There are people who are better or smarter than them. And if they’re not as smart as those people, then they won’t be able to make contributions, and they won’t be able to do good work. And in my experience, having worked with many students over the years, but also having read a lot of history and history of science, we’ve discovered that it’s really true that we’re all, we’re all points in some very high-dimensional parameter space we’re not ranked. And knowing how to leverage your unique profile of strengths and weaknesses to make progress in the work that you find meaningful, knowing how to do that is part of what it means to enter an academic field and grow and learn. And develop. So I would say don’t worry so much about where you think you fall in the ranking and focus more on figuring out how to bring your unique profile to the table and how to use it to make progress on the things that you find meaningful. That’s probably my best advice. And for the researchers watching, not advice, but any message you would like to see, well, you can give advice as well. Interdisciplinarity is great. We should have more focus and spend more money on places where disciplines intersect. I work at the intersection of physics and mathematics and philosophy and now, you know, statistics, you know, dynamics. There are many different fields that I meet in the place where I’m currently working. And I think this isn’t new, but I think some of the most exciting work that’s happening in scholarship in general and certainly in science or philosophy in particular is happening at the boundaries of multiple disciplines. So I would say that maybe not for the researchers, but for the people who fund the researchers. Yes, spending more time thinking about how to foster work at the boundaries of disciplines where there’s a real cross-pollination between disciplines, I think would pay huge dividends to progress in all of our work.

I agree. Last night, one of the reasons for my being here besides seeing you was at MIT, there was an event by Adi here yesterday, last night, called “ekkolapto,” which was about neuroscience, perception, biology, physics, language. Yes, it was a mix of all kinds of major fields in science and technology and engineering and mathematics (STEM) and the arts where they intersect and can be used to create something really useful. But you have to be rigorous about it. You can’t just, you know. So that was fun. Anyway, thank you. It was my pleasure and I know you’re eager to get going and I appreciate your students have been waiting for that. That’s great. I appreciate your time. Thanks for having me. That’s great. I really appreciate the invitation and all the work you’ve done in this wonderful podcast series. It’s wonderful work talking to the audience and talking to people who work in all these different fields about their work and clarifying complicated topics for everybody, I think that’s wonderful work. Thank you. And next time the interview we do will be more technical. We can even use a whiteboard. We’ll see. We’ll see. Anyway, it’s been a joy. Thank you so much for the invitation. Thank you. I also thank our partner, the Economist. I also discovered last year that outbound links are very important in the algorithm, meaning the more you share on Twitter, for example on Facebook or even on Reddit, et cetera, it shows YouTube, hey, people are talking about this content outside of YouTube which in turn helps significantly with distribution on YouTube. Second, there’s a remarkably active discord and subreddit for Theories of Everything where people explain toes. They respectfully disagree about theories and build as a community our TOE links for both in the description. Third, you should know this podcast is on iTunes. It’s on Spotify. It’s on all the audio platforms. All you have to do is type Theories of Everything and you’ll find it. I personally benefit from rewatching lectures and podcasts. I also read in the comments that TOE listeners also benefit from replays. So, how about a relisten instead on those platforms like iTunes and Spotify and Google Podcasts, or whatever podcast app you use. And lastly, if you’d like to support more conversations like this, and more content like this, consider visiting patreon.com and slash CurtJaimungal and donating whatever you like. There’s also PayPal. There’s also crypto. There’s also just joining on YouTube. Again, keep in mind that support from patrons and from you is what allows me to work on TOE full-time. You also get early access add-free episodes, whether audio or video. It’s audio in the case of Patreon, and video in the case of YouTube. For example, this episode you’re listening to now was released a few days ago. Every dollar helps far more than you think. And in either case, your viewership is karmic enough. Thank you very much.