There’s No Wave Function? | Jacob Barandes [Part 1]

I didn't know it was impossible. Therefore, I brought these theories closer together, expecting that there would be a moment when I would need to make some kind of jump to move from one theory, classical physics, to the other, quantum physics. And the strangest thing happened. The faces of the abyss merged. And there was no longer an abyss.

Today I have a special treat. I traveled to Harvard University to meet Jacob Barandes in person, the co-director of graduate studies in physics and the person who has caused a stir at the intersection of fundamental physics and philosophy. In his unassuming office, he was working on a revolutionary framework that finally explains quantum mechanics. His radical proposal suggests that there is no fundamental wave function, that Hilbert spaces are not real, and there is no mysterious quantum world. It's just a mathematical convenience. It's a tool of the theory. But there is nothing in physical reality that it describes. In this episode, we delve into the picture that solves the measurement problem, eliminates the need for multiple worlds, and finally gives a clear vision of what actually happens at the smallest scales of nature. However, this clarity comes at a price. The question is: Are physicists ready to accept quantum mechanics without magic?

Professor Jacob Barandes, welcome to Theories of Everything. I've been following you for a while, stalking you online in a non-creepy way, and we've been talking off-air about physics. However, you're also the co-director of graduate studies here in physics, and you have a fascinating background in how you got into physics. So why don't we start there? Why don't we start at the beginning?

From the very beginning, my first intellectual passion was the philosophy of mind. When I was five years old, I was sitting in one of those little assembly rooms that you have in schools, you know, like my elementary school, and one of the teachers was talking loudly about something. I don't remember exactly what they were talking about, but I remember sitting on this soft carpet, and there was a piano, you know, the piano legs look like lion's feet. And suddenly I became aware of my existence in a very clear and explicit way. And I was utterly perplexed, and I tried to express my state of perplexity to one of the other children sitting next to me, and I utterly failed to be able to explain what I was feeling at that moment. And so this left a kind of mark on me. And from that moment on, I was very interested in questions concerning, you know, what is the nature of our existence? What does it mean to be conscious? So the philosophy of mind is where things started for me. But of course, I didn't have the language to express any of these ideas.

I loved science documentaries. I grew up in New York City, and my parents would take me to the American Museum of Natural History in New York City, which is a truly wonderful and magical place. And anyone who goes to New York City should really visit it. It's amazing. So I became really interested in science. I became interested in mathematics. And I thought that if you're interested in science and mathematics, and you're interested in foundational questions, that means you should go into physics, right? That's where I thought the mathematics and these foundational questions, where they come together. So I went, you know, through school. I got to high school. I first studied physics in an academic way in high school. I loved it. It was my favorite class in high school. I still have all my old notebooks from that time. It was the only class where I was excited to go home and do the homework, and that was great. My teacher, Doug Bartell, was exceptional. He was absolutely amazing.

Around that time, I also started doing return trips to New York City. We had moved out of the city at that point, about an hour north. And so I would take the train and go into the city on weekends. And I would take science classes in the city. Columbia University has a program called the Science Honors Program. At that time, it was run by Alan Blair, who was a wonderful human being who was a physics professor there and also ran this special program, the Science Honors Program. And these were classes you could go to. There's no homework or grades or tests. You just come on a Saturday morning, and you will learn all sorts of amazing things. It was an amazing program. And I would also work at the American Museum of Natural History. So I started interning there when I was in high school. And I started working in the astrophysics department. I wasn't an expert in astrophysics in high school, but I was very curious about it. And I was doing a lot of computer programming at that time. And they were interested in having me work on some projects that they were doing. They were working on computer graphics visualizations of the galaxy. I worked on a project called the Digital Galaxy Project, if I remember the name correctly. And that was my first opportunity to work with scientists. And I didn't know any scientists. We didn't have any scientists in our family. My father's a doctor. But research scientists, I had never met anyone like that. I didn't know academics or professors or anything like that. So these two experiences were very important for me. And with the physics class, I decided that I wanted to study science and maybe physics. But my love for the philosophy of mind was still very strong. And so when I went to college, I decided that I would try to study the brain. I loved computers. And so the idea was that I would somehow find a way to combine my interest in computers and neuroscience somehow. And in order to pursue that direction, I had to take some basic science classes, including physics, and I had to take math classes. And I really fell in love with those classes. I started studying biology in my sophomore year. And I enjoyed it, but not nearly as much. And I have a lot of respect for people who work in biology, but that wasn't for me.

Yes. For me, I just want to know if the reason is the same for you. For me, it's just a multiplicity of terms. And I feel like it's memorization more than derivation. Yes, I sometimes feel that way. I mean, I want to make it clear that I'm not a biologist. And I don't want to, you know, make any assumptions about the way biologists think about their subject. I know there is internal logic to biology. Oh, I just mean that from the freshman level, at least from what I went through, the older I got, or the more experienced I became in research, maybe it's different. I had this experience when I took the first semester of biology, and it was microbiology, and it was very logical. Then the second semester was anatomy. And that was much more memorization. And that I went through the same experience you describe, it wasn't really for me. You know, but it was hard to convince myself that I should study physics because, you know, there were all these extremely smart young people who study physics and mathematics, and I was interested in the theoretical side, the mathematical side of physics. And I didn't feel like I was good enough to keep up with people. But in my sophomore year of college, you know, I had conversations with some really wonderful people, some of my fellow college students. And basically, I decided, you know, that it's very important to do what I'm excited about, and then, you know, I want to be the best at something. So I'll do it, I'll do it anyway. And so I just started taking a lot of physics and math classes. And it was great. It was an amazing experience. And some of the philosophy courses that we were required to take at that time didn't really appeal to me. There was a lot of focus on ancient Greek philosophy and early modern philosophy. Now, I appreciate those topics, but, you know, they weren't as exciting to me as what I was doing in physics, at least at that time. Now, I have to say that, you know, as the years went by, and I started exploring other areas of philosophy that weren't part of the standard curriculum in college, analytic philosophy, philosophy of science, logic, foundations of mathematics, philosophy in physics, my views about philosophy completely changed. But, you know, at the time, I wasn't, I didn't have the opportunity to see much of that. And I didn't see it grow up. And so I thought, okay, I'll focus my attention very much on physics. So I finished my undergraduate degree, and then I went on to graduate school. And I started my PhD here at Harvard in theoretical physics. And as I progressed through that degree, you know, I started thinking again about some of these old philosophical questions that I had. And I did my PhD in high-energy theoretical physics. And, you know, I think we did really good work. But I was actually more excited about the philosophical questions. And by the time I finished the PhD, I knew that this was the direction I wanted to go in. So I continued working in the department. I'm a lecturer here in the department, and I also co-direct the graduate program as I think we talked about. So, you know, I mix teaching physics, doing research, and advising students. It's been a really rewarding experience. The students here are amazing. They're ideal, and they're wonderful. And they're profound. And I've really met some of the most wonderful people anyone could ask to meet here. I've met lifelong friends here. I mean, it's just been an amazing experience being here. And I think I've learned a lot, far more than any of them have learned from me. But this is how one who works primarily in philosophy, philosophy of physics, finds themselves embedded in a physics department. So I have a faculty affiliation with the philosophy department, and I rotate teaching physics classes there as well.

Yes. But my research is primarily about the intersection of philosophy and physics. So what happened to the philosophy of mind? That's a good question. I'm still really interested in the philosophy of mind. I don't think my tool set is what I think, you know, we all come into this world with a distinctive profile of strengths and weaknesses. I'm very interested in the philosophy of mind, but I don't think my distinctive profile of strengths and weaknesses is optimal for working in the philosophy of mind. So I read about it. I'm intensely interested in it. But I feel that I'm better suited to the questions, the intersection of physics and philosophy. And that's where I've spent, I've spent my time and my time and my focus. You said earlier that you don't want to pursue something just to be the best at it, but you want to pursue your interests. But you're saying here, okay, my interests still lie in philosophy of mind, but it's not that, I'm not the sharpest tool in the shed when it comes to that. That's not my skill set. How do you reconcile those? That's good. Yes. I mean, we all change as we get older, right? Yes. So I think for me it's been about finding a balance. It's less important to me. And I, you know, I've talked to students about this as well, right? It's less important, you know, what you want to be. It's not important to be big or famous or important. And for me, what's more important is doing meaningful work, which I feel is meaningful, which the person feels, is meaningful work we feel is meaningful, but we also feel like we're making progress. So it's not that I don't want to work in philosophy of mind because I think that won't make me famous. My lack of motivation to work in philosophy of mind, at least now, and this could change, is because I, at least now, don't feel that this is where I'll be the most productive. It's where I feel that I won't feel that I will do work that I felt proud of like the work I do in philosophy of physics and philosophy of science. And that's why primarily I focus on these areas now.

I'd like to get you to justify the non-existence of a wave function pretty soon. However, I want to know if there's a relationship between the philosophy of mind and quantum mechanics first. Yes, that's a very interesting question. And in fact, this will lead us to a very important research paper which I hope we can talk about from the early 1960s. It was a paper written by Eugene Wigner. I think the paper was called The Question of Mind and Body. The title might be slightly off. But Wigner presented a thought experiment that I think is in some ways related to his view that there is something linking consciousness and the collapse of wave functions. This thought experiment actually goes back earlier. It's not entirely clear to me who originated this thought experiment because it first appeared in Hugh Everett's extended dissertation. Hugh Everett was a graduate student at Princeton and was at Princeton at the same time overlapping with Eugene Wigner. And Hugh Everett originated the many-worlds interpretation of quantum mechanics and wrote this very long dissertation. It's 137 pages. I think that's maybe not long in the absolute sense. And in 1956-1957, he didn't end up publishing it. And his advisor strongly encouraged him to publish a different, shorter dissertation. But you can find a copy of this extended dissertation and it opens with the same thought experiment. So we know that this thought experiment was presented before Wigner by at least seven years before Wigner's paper or something like that. But that didn't get published. Wigner's paper was published and so it's widely attributed to him. Wigner conducted this thought experiment that I'll talk about in more detail if you'd like because I think it gives a good motivation for why one might think there's a problem with the way we formulate quantum mechanics at a textbook axiomatic level. So I'll come back to it. But I'll just say that this thought experiment was apparently related to Wigner's idea that consciousness is somehow linked to the collapse of wave functions. And I think this creates a potential link between the philosophy of mind and quantum mechanics. But there are other connections. In the many-worlds interpretation, which is a formulation of quantum mechanics that Hugh Everett presented in the mid-1950s, one encounters some fundamental questions about consciousness and human identity in a universe in which a giant universal wave function splits into parallel realities, each of which contains copies of yourself. How do we understand this kind of picture and make it relate to the way we think about fundamental problems in the philosophy of mind? I don't claim to have an answer to this question. I think it's an ongoing puzzle. So I think there are some interesting links between the philosophy of mind and quantum mechanics. And I think my general position has been anytime you're unable to answer a question about quantum mechanics, because you've somehow linked it to the philosophy of mind, and in particular to an unanswered and maybe unanswerable problem about the philosophy of mind, I'm deeply concerned.

Okay, let's talk about the wave function, because when we talk about the collapse of the wave function, the collapse of something, what is the wave function? And why do you think there is no such thing? That's a great question. So let me say a little bit about some history. I think some history here might be helpful in setting things up. But before I talk about the history, let me first answer your question. I'll give a short answer, and then we'll talk a little bit about history, otherwise we might get lost in the weeds. There's a common story about quantum mechanics that many students coming into this field hear. The first fundamental object in quantum mechanics is the wave function. It's this mysterious wavy object. It's a little unusual. It's kind of like the electric field, except that its values are complex values. They're complex numbers. They're like ordinary real numbers plus a number times the square root of minus one. And somehow, they're related to probabilities in some way. And somehow, the object of quantum mechanics is wave functions. They're the central player in the story. And the way you think about quantum mechanics starts from the way you think about wave functions, right? And they're the heroes of this story. I think there's some good reason to be a little skeptical of that picture. And I don't think there's much really — there'd be a lot of controversy about saying — I think, you know, as you learn about quantum mechanics and you start to see that there's a much bigger picture, you know, you start to see that the wave function in this kind of story is an oversimplification. I think that if you pressed a physicist hard enough, they would actually admit that this picture is a little — it's a little quick, and there's a deeper picture underneath. And so I think one way of thinking about what I'm saying here, your question, is to take those concerns and take them to their logical conclusions, right? We ask ourselves, well, what is this theory really about? Is it really about wave functions at all? And in that regard, are wave functions things that describe things that actually exist in the world? And this is where one lives in a kind of borderline space between physics and philosophy when you ask questions like that. On the one hand, it's about physics. It's about, you know, what's the right way to think, and what's the right way to formulate, you know, what can be our best and most successful scientific theories, and certainly when it comes to numerical accuracy and precision. You know, and on the other hand, what's really there? What's in the world? What exists, right? That's the ontological word that metaphysicians use. Ontology is the study of what truly exists. And let's ask, does the wave function describe something physically existing? That's a question. To explain a little bit about how we got here, I can say a little bit about the history, but let me pause and ask, do you have any follow-up questions on that? I want to make sure, you know, to make sure we all have time for that.

Certainly, certainly. So when you say that there's the standard picture of the wave function, you said, okay, what people learn is that there's a wave function, and it's related to complex numbers, and it's also related to probability in some way. Is that the picture you mean? Or are you referring to the picture that represents the wave function as something in physical space and being of something? Yes. So when I say that, what I mean is, when people say wave function, when physicists talk about the wave function, it's the wave function of something, the wave function of the electron, the wave function of so-and-so. So what is that picture that you're about to smash exactly? Yes, that's an excellent question. I think, you know, the problem here is that it has a little bit of both, right? The wave function plays a functional role in quantum mechanics. That is, it does certain things for you theoretically. And we study its change with time. There's this famous equation called the Schrödinger equation, which roughly says that when the system is left to itself, it's not disturbed, it doesn't exchange information with any other systems. Then it has a wave function that changes with time in a smooth way. And this smooth process is described by a differential equation, which is roughly an equation that just tells you what the wave function is at each infinitesimally later moment, given what it was at the previous moment. And this equation is known as the Schrödinger equation. So we plug in the initial wave function, and we use the Schrödinger equation to find out what the wave function will be, how it will behave as time goes by. And then there are rules for taking the wave function and calculating predictions. So that's uncontroversial, and I certainly agree with that picture, that we should use wave functions for that purpose. But I think it's easy to let that picture lead one to believe that wave functions describe physical things that exist in the world. And, you know, when I think students arrive at a quantum mechanics class for the first time without having studied the theory, they might think that, well, every particle has some wave function, and wave functions are kind of like, they're kind of like waves. And, you know, empty space is kind of like the ocean, and all the particles are like little waves moving in the ocean. And that's not actually the picture we get even from the textbook theory of quantum mechanics. In the textbook theory, the wave function is an assignment of a complex number to every possible configuration of your system. If you're talking about one particle, nothing else, then every possible configuration is every possible location the particle could be. And each location is distinguished by three numbers. They're the x, y, and z coordinates, if you like. And so the wave function takes these three coordinates and assigns to it a complex number, and then we're supposed to do a mathematical operation with this complex number, and the probability comes out. More precisely, the probability density, but the probability per unit volume that a measurement will find the particle at that spot. So you can think of the wave function as assigning a special complex number to every point in three-dimensional physical space, where you do this operation, which is called mod-squaring, and you do this operation, and then it tells you the probability, roughly, by which a measurement will find a particle at that location. So it's very easy to believe that the wave function is like a field, like the electric field, in three-dimensional space. But if you have two particles instead of one, the probability space is more complicated. We call this probability space configuration space. Because every possible configuration of a two-particle system requires six numbers, not three. You need to know the x, y, and z of the first particle, and you need to know the x, y, and z of the second particle. So the wave function assigns a complex number to a point in six-dimensional space. Six dimensions because you need six numbers to specify its points. This means that the two-particle wave function is a function, we say, its domain is six-dimensional space. Six-dimensional space is not three-dimensional physical space. If you have three particles, it's nine-dimensional space. If you have ten particles, it's thirty-dimensional space. And in fact, our universe doesn't have a well-defined number of particles at all. Our leading physical model of the universe, at least for the non-gravitational parts of the universe, is the standard model, which is based on a collection of models known as quantum field theories. And in quantum field theories, particles are emergent excitations of these kinds of delocalized entities called quantum fields. And the number of particles can change from one moment to the next. It's not always well-defined. So it's not even clear how to think about wave functions that live in anything like physical space in the universe as we know it. So I don't mean to say we teach our students incorrectly. We don't. I mean, students will learn this as they progress through their physics trajectories. But I think many people on the outside or new students who haven't yet started their own journey in physics have a certain idea about what wave functions are that is actually quite different from the way we use them in practice. I don't know if that clarifies what you were asking about.

Okay, you have a lecture about how Hilbert spaces aren't real too. I'd like you to talk about that. Now, I understand you said more precisely that Hilbert spaces are superfluous in the same way that measurement symmetries are superfluous or that measurement transformations are superfluous. So at this point, if it's okay, let me talk a little bit about history, if it's okay. So where does quantum theory begin? The lightning, like grand tour of quantum theory. There are these actual physical things called black bodies. They're rooms that are heated. And, you know, there's a little hole in them. I mean, you don't really poke holes in them. It's more complicated. I'm largely a theorist, and my visualization of experimental physics is that it should be better. And I apologize in advance to any experimentalists listening. But these were actual systems. People actually built these things. These were rooms. They would heat them up. And then they would have a little hole in the room through which they could see the radiation coming out of them. And they could put this radiation through experimental apparatuses that could tell them how intense the radiation was as a function of wavelength. We all know that different wavelengths of light correspond to different colors in the visible range. And if you go outside the visible range, you have very long wavelengths infrared or, you know, microwave radiation or radio waves. And at very short wavelengths, you have X-rays and so on, gamma rays go out, and ultraviolet, and gamma rays go out to very short wavelengths. And you can ask yourself, if you plot how strong the radiation is as a function of wavelength, what kind of pattern should I expect to see? And the pattern that was revealed in experiments was difficult to explain on the basis of a theory from first principles. A very important physicist, Max Planck, in 1900, discovered a way to generate a theoretical prediction for what the blackbody radiation curve should look like. And it agreed with experiments. But in order to get there, he had to kind of fudge his formulas. He had to introduce a fudge factor. He had to assume that the radiation in this room could only occur in quantized amounts. This means that there were different wavelengths you could produce, but each wavelength could only be excited in discrete steps. This led to

The quantum hypothesis. This was his quantum hypothesis. There was some kind of parameter that tells you how much you want to implement these steps. Today, we might call it the regulator. And this parameter, we now call it h, lowercase h. We call it Planck's constant. Planck introduced it, and I think his idea at some point was that he wanted to introduce an intermediate step in the calculation, and once he calculates everything, he sends this clearly, in his mind, non-material parameter to zero. But whenever he tried to send it to zero, he got wrong results. He realized that this parameter was not zero, but it was very small. It was, today, it has a value of the order 10 to the minus 34, and 10 to the minus 33 in conventional units, and he couldn't get rid of it. And so people had to accept that there was this fundamental distinction in nature that he couldn't explain.

Then for the next 22 or 23 years, the period from 1900 to 1922, 1923, physicists working on these questions were living in a time we call today old quantum theory, a model we call old quantum theory. Old quantum theory was built on a mixture of physical pictures of particles moving in space, in orbits, as in atoms, and then a set of ad hoc formulas and rules that people didn't quite understand that seemed to capture some of the observations that were coming out of experiment. But it was a very murky time. There wasn't anything like a fundamental theory from which this whole picture emerged. And the formulas weren't perfect, and they didn't quite work properly.

In 1913, Bohr proposed, as in Niels Bohr proposed his famous model of the hydrogen atom. This is a model where you have the nucleus, and for the hydrogen atom it's just a proton, a positively charged proton in the middle, and an electron orbiting it. And by making use of this kind of heuristic formulas that were characteristic of that time, Bohr argued that the electron could only be in certain specific orbits, and when it transitions between them, its energy changes in discrete steps, and whenever its energy changes, it either absorbs or emits radiation in discrete amounts. And with this, he was able to explain the specific colors of radiation that we can see emitted from excited hydrogen atoms, or absorbed by hydrogen atoms, what are called line spectra. And we found that other atoms have line spectra too. Helium, famously, was discovered because we looked at the spectral lines of the sun, and after we accounted for all the spectral lines that we knew about, there were still more lines that we hadn't accounted for, and so people guessed that there was a new element. They named it helium after Helios, the sun, and eventually we found helium on Earth, but it was first discovered in the sun. Clearly this was a very big deal, and Bohr ended up winning the Nobel Prize in 1922 for this Bohr model.

But by about 1922, at least by that time, people were getting very skeptical, because people couldn't capture these clear pictures of particles moving in specific ways, particles interacting with fields like the electric and magnetic fields, which were well understood by this point. They couldn't capture this picture of the world, this ontology, this picture of what was physically there. They couldn't find the laws that when combined with this picture gave you the correct predictions, that gave you sufficient empirical meaning, empirically sufficient, a theory that was able to explain the things that we were seeing, and make predictions that were regularly confirmed.

Niels Bohr gave a series of lectures in 1922, and Heisenberg, Werner Heisenberg attends one of these lectures, at least one of them, and the story goes on, and I'm not entirely sure how much of the story is apocryphal or how much actually happened, but the story goes that Heisenberg at some point objected to this lecture. Now Heisenberg was 20, maybe 21 years old, and Niels Bohr was a Nobel Prize-winning physicist, widely considered to be the whisperer of quantum, someone who intuitively understood quantum mechanics in a way that nobody else did. And here was Heisenberg challenging Bohr. And I think it was expected that people would think that Bohr would be upset about this. But the story goes that after the lecture, Heisenberg went for a long walk with Bohr, for many hours, and that had a profound effect on how Heisenberg thought about nature. The story goes, the legend goes, that Bohr revealed to Heisenberg that, in Bohr's words, he wasn't convinced that there were particles or orbits at all.

You start to see comments from people like Sommerfeld, and Arnold Sommerfeld was actually Heisenberg's PhD advisor in Munich where Heisenberg was. And then people like Wolfgang Pauli, many of the people who founded this developing theory were increasingly skeptical that this world picture could survive. Then in 1925, Heisenberg, who was visiting the leading institution in mathematics and theoretical physics at that time, Göttingen, where there were people like David Hilbert, and there was Max Born, and Felix Klein, leading mathematicians and physicists. Heisenberg was visiting there, and he was visiting Max Born’s group, Max Born and his student Pascual Jordan. And he was thinking about all these questions and was terribly puzzled by all of them. There were all these new more complicated formulas that people were writing down that were connecting energy levels and transitions and atoms to spectra. And he got this terrible case of hay fever. In the spring of 1925, he went to Helgoland, which is this island where the pollen levels are much lower. And he comes back with this completely bizarre draft of a paper. And he gives this paper to Max Born and Jordan, and then goes on vacation again. He goes on a vacation of sorts. He’s exhausted from all his intellectual efforts. And this paper begins with these remarkably striking statements, statements that a philosopher of science would immediately recognize as pointing to a kind of paradigm shift. Heisenberg says at the outset, we should not think about orbits or particles anymore. We should completely reformulate our physics, our theories, in terms of quantities that are in principle measurable. And so he formulates this different way of thinking about quantum mechanics in terms of abstract objects that Max Born identified as matrices. People listening to this might have heard of a matrix. And the subject to which they belong, which is linear algebra, was not a fundamental part of the physics curriculum at that time. In fact, Heisenberg independently discovered matrices in this work. And their computational operations. And Born recognized what Heisenberg was doing. Then together they wrote two papers where they presented this thing called matrix mechanics. In matrix mechanics, there is no picture anymore. There is no picture of atoms and electrons and particles orbiting atoms. There’s just this abstract mathematical apparatus for predicting the energy levels of things.

Schrödinger comes along a few months later, and through a series of theoretical arguments that I won't be able to do justice to here that connect to what's called Hamilton-Jacobi theory, a beautiful area of classical physics. Heisenberg proposes a different way of thinking about quantum mechanics. He introduces his wave function. And in terms of which appears in these papers. Schrödinger introduces his wave function. This was in early 1926. And in some papers in German and in English he wrote in both languages. He called this theory the wave theory of quantum mechanics, which is a beautiful word, isn't it? Wave theory, it’s a great way, beautiful term. And he introduces his wave function, and as he handles it he treats it as a mechanical object. He immediately knows it lives in configuration space. He says this in his early papers, but he says maybe this is reality. Maybe reality is a giant undulating mechanical wave in configuration space. And through some mechanism that we don’t quite understand, this undulating wave in this abstract high-dimensional space somehow projects the facts into three-dimensional reality, facts about where electrons are. He doesn't have the complete picture here, but this is essentially what he describes. And he’s able to use this wave function as an indirect way of predicting energy levels. Max Born comes along very shortly thereafter and proposes that wave functions are not mechanical objects. They are mathematical tools. The reason they’re appreciated in a complicated way is that they aren’t really physical things. They’re things that we work on using these kinds of mathematical operations to generate probabilities. Wave functions, when you take the complex numbers that describe them and you perform this operation on them, what comes out are probabilities. The wave function should be understood as a mathematical apparatus for generating probabilities. And by 1928, Schrödinger had already started to back away from his view. He gave a lecture in 1928, his fourth lecture on wave mechanics, where he said that he used to think that wave functions were physical things. He even foreshadows, in some ways, the many-worlds interpretation. And he says it’s an object where everything that the system can do plays a role in this giant wave function. But he didn’t believe this anymore. He had accepted what seems to be the new idea.

And then over the next couple of years, 1930, Paul Dirac wrote a textbook summarizing all the quantum mechanics that was known at that time, and he connected Schrödinger’s wave functions with Heisenberg and Born and Jordan’s matrix mechanics. And he realizes that they are all part of this deeper reality. There is this mathematical structure. It’s a kind of space called Hilbert space, it’s a kind of vector space. But it’s a vector space that contains complex numbers. And when you look at the vector space one way, you see wave functions. If you look at the vector space another way, you see matrices. You see what Heisenberg was doing. They’re all parts of this mathematical structure which is called Hilbert space. And then two years later, John von Neumann, the great mathematician, wrote a book called Mathematical Foundations of Quantum Mechanics, where he formulated this theory in rigorous mathematical terms. And that’s what we’ve had ever since.

Today, when people refer to the Dirac-von Neumann axioms or the axioms of quantum mechanics, they mean this recipe for generating predictions about what we will see in experiments, empirical predictions. And this framework relies on Hilbert spaces. Roughly, the first axiom says that every quantum system has this abstract space called Hilbert space, which is not configuration space. It’s a completely different kind. And that the state, to some extent, of the quantum system is represented by certain kinds of objects in this Hilbert space. State vectors, in the simplest cases, are density operators, somewhat more complicated. The second axiom is that when the system is left to itself, it evolves according to Schrödinger’s equation, generally, unitary evolution. Then there are three other axioms that talk about when a measurement is made on the system, and what you are likely to see. These are what are called the measurement axioms. One has to do with the kind of mathematical objects that represent the things that we can measure. These are called observables. One has to do with how to generate probabilities for what you will see when you make measurements. This is called Born’s rule. And the last one is that after the measurement, the state vector of the quantum state collapses. This is how we present quantum mechanics to students in textbooks now. If you go to my shelf and look at any of the textbooks on quantum mechanics, it will present something like these five axioms. Different people will sort them out differently. They might not say there are five. They might say there are four or six or some other number. But I like to divide them into five, don’t you? So we know we have these axioms. We know they’re very good at giving us a prescription for generating empirical predictions. But what do they tell us about what’s actually happening in nature? Do they say that the seat of reality is actually an abstract Hilbert space, or a very high-dimensional or infinite-dimensional vector space over the complex numbers? And that the universe is some object in that space that evolves according to some rule.

There are some reasons to be skeptical about this. The first is that we can reformulate the axioms of quantum theory in a completely different language. So people working in different other areas of quantum physics, in mathematical physics, sometimes prefer to formulate quantum mechanics in what’s called C-star algebras. C-star algebras aren’t things you find on the beach. They aren’t that kind of C-stars. But they are abstract collections of mathematical entities that encode how systems evolve in quantum mechanics, and how to generate measurement predictions in a way that doesn’t begin with Hilbert spaces. When you take this alternative point of view, you get a very general mathematical framework from which the Hilbert space picture can emerge certainly, well, you can almost always pull out the Hilbert space picture in these stories. Sometimes you get a unique Hilbert space. Sometimes you get a proliferation of different Hilbert spaces, and there are all kinds of deep questions about what that means. But when I started to learn about the algebraic formulation of quantum mechanics of quantum mechanics, it opened my mind, as I think it does for many people, to the idea that maybe the Dirac-von Neumann axioms, the ones that we hear about in textbooks, which are primitively formulated in terms of Hilbert spaces, might not be the end of the story, and that Hilbert spaces might not be the fundamental way to think about the structure of quantum mechanics. Now, I think one can go much further than that, and maybe we’ll talk a little bit about how one can go further. But once you actually start to see that there are mathematically equivalent ways of formulating quantum mechanics that don’t begin with Hilbert spaces, that makes you somewhat skeptical that the Hilbert space picture is fundamental, that the objects that live in the Hilbert space picture… the state vectors that become wave functions when viewed in a certain way, are fundamental objects.

Well, what allows you to say that Hilbert spaces that can come from a C-star way mean that Hilbert spaces are more illusory? Does C-star algebra allow you to calculate something that the Hilbert picture doesn't allow, but it allows you to calculate everything that the Hilbert picture does? Does it therefore encompass it? So that’s an excellent question. In the standard approach to quantum mechanics, the first three measurement axioms say that everything observable you might want to look at is represented by a kind of matrix, and more technically, a self-adjoint operator in Hilbert space. It’s kind of like an abstract mathematical entity that you can add other such entities to and multiply them together. It acts like a number, like the variable x in algebra, but it’s a little bit weirder, in that they don’t always commute with one another. They’re kind of weird objects, but we have a collection of mathematical tools for taking them and creating empirical predictions. When you start with the textbook axioms, you start with Hilbert space, and then these operators are the things that come out of Hilbert space. In the C-star algebra approach, you go in the opposite direction. You start with an abstract specification of the things that are observable, which are represented by these abstract mathematical symbols. And the symbols have all kinds of mathematical properties, rules for how to add them together, multiply them, just as I said, but they didn’t start as things living in Hilbert spaces. They have their own existence, their own identity. And there’s also a funny way that people working in this area sometimes talk about it, where they will say, are you a Hilbert space conservative? Meaning, do you consider Hilbert space to be the starting point, or are you an algebraic imperialist? Do you consider the algebraic objects to be the fundamental ingredients? When you start with these fundamental algebraic ingredients, they form a special kind of mathematical structure called a C-star algebra. The C refers to closed, meaning there are no holes in the algebra. Everything is filled in all the way. And the star just refers to the complex conjugation operation, but on these objects, roughly. And there are some other properties that it needs to possess, but they’re very primitive properties. And what you can then show is that there’s a way to construct a Hilbert space from these algebraic objects. Once you’ve done that, then they become matrices or operators in Hilbert space. But once you see this construction, it’s called the Gelfand-Naimark-Segal construction. It’s a beautiful theorem. Once you’ve done that, you start to wonder how important Hilbert spaces are, especially since some C-star algebras don’t generate a unique Hilbert space. And now you have this question about exactly which Hilbert space you’re supposed to use. And that’s where the Hilbert space multiplicity comes from. In a way, yes. So for simple systems, if you have one particle or three particles or 10 particles, it turns out you essentially get a unique Hilbert space. So you can really think of either one as fundamental. But when you think about systems that have an infinite number of moving parts, any system that you design as if it has an infinite number of particles, we do this in thermodynamics, or in a quantum field where there are variables at every point in all of space. For that kind of system, you don’t necessarily get a unique Hilbert space. Then there’s this question which is more fundamental. And this is an ongoing debate right now. It’s called the problem of inequivalent unitary representations of C-star algebras. And it’s an ongoing puzzle in the foundations of mathematical quantum mechanics.

Earlier we were talking about Heisenberg and how he started with observables, and Schrödinger didn't. And now you’re talking about taking those observables seriously and forming an algebra out of them only called C-star algebra. But then you also said you can take representations of the observables, which become matrices. Yes. And Heisenberg was dealing with matrices. Exactly. Right. Yes. Your picture is exactly correct. So this is not what I work on now. But when you start to realize that you can reformulate the rules of quantum theory in a different way, that kind of opens your mind to other possibilities. I want to know what opened your mind to. Yes. Where is your mind these days about quantum? That’s an excellent question. So let me go back to the Wigner’s friend thought experiment again, because I think that’s a good link to where I end up, and where I think a lot of people end up. So what was this thought experiment all about? In a way, I mean, it’s a thought experiment known to many people in physics and certainly philosophy of physics. It’s a really good workhorse in quantum foundations. But I think there are still a lot of lessons to be learned from it. And people have extended this thought experiment in many different directions. There are what are called extended Wigner’s friend scenarios that many people are working on. But I think the original version of it is really great. And as I said, it was formulated way back in 1956 in Hugh Everett’s original thesis. But it’s a very simple question. The axioms say we have a Hilbert space and we represent the state of the system as some object in this Hilbert space. The questions that we can ask are represented by observables by these symbols. We can generate probabilities. Then after the measurement we collapse. Why do we collapse? What is the purpose of the collapse? The collapse is to ensure the strength of the measurement outcomes. After making a measurement, you are guaranteed to get the same result again. And that’s what the collapse achieves. It guarantees that once you’ve got a definite outcome, if you make the same measurement again before you give the system any chance to evolve any further, you’re almost 100% guaranteed to get the same result. Now, the problem of course is, well, what is a measurement? What kinds of physical processes count as measurements? One point of view is that we don’t need to worry about this. One view is that all scientific theories are ultimately utilitarian projects, meaning they exist only to allow us to connect preparations and settings of measurements and predictions of the measurement outcome or outcomes of measurements. That’s all a scientific theory aims to do. Doing anything else, drawing pictures of reality is just fluff. It’s like mythology. It’s not necessary. It’s philosophy. It’s metaphysics. It’s not important for physics to do that. And the theory does that. The theory says that we make a measurement. And here’s what comes out. The theory does what it’s supposed to do. What more could you ask for? The theory says when we make a measurement, we do this thing, we apply this recipe, we collapse the quantum state of the system, and then we move on. Everything works out fine. That’s your problem.

The Wigner’s friend thought experiment considers observers. The first is Wigner, in the outside world, enjoying a nice sunny day, maybe a beautiful autumn day like today. And Wigner’s friend who is inside a completely sealed box. Now, people might wonder in quantum mechanics, can you really seal anything in a box? Can’t things leak out? Well, particles can leak out of certain kinds of containers via tunneling, but if you make the box thick enough, impenetrable enough, in a very idealized sense, then you can keep everything inside the box, at least for the duration of the experiment. Wigner’s friend is inside this box with a quantum system, some system in a superposition of two possibilities. Let’s say it’s a particle that is spin up and spin down mixed together in some superposition. Wigner’s friend engages in some interaction that we might call a measurement of this superpositioned quantum system that’s inside this box. Wigner, on the outside of the sealed box, doesn’t make the measurement, and doesn’t in any way know what happened. Now we have two observers, and now we have a question. Do we activate the collapse postulate or do we not activate the collapse postulate? We have two observers, and we now have to take a stand, and we have to say whether or not we’re going to do it. And there are only a few possibilities. One possibility is yes, we activate it. We activate this measurement axiom, and the quantum state of the system collapses, even though Wigner on the outside hasn’t seen anything, it just collapses. For everybody, it just collapses. But now we have a question. Wigner on the outside, why does Wigner think it collapsed? Because Wigner’s friend made the measurement? But how do we know that the thing that Wigner’s friend did qualifies as a measurement? Wigner’s friend is just a physical system in the box. If Wigner’s friend is replaced by an electron that interacts in some way with the other particle, we wouldn’t call that a measurement. If Wigner’s friend is replaced by a tardigrade, a water bear, which is one of those extremely resilient microscopic creatures that you can barely see with the naked eye, would that have been a measurement? Now we have just a physics question. At what point do we cross the line to yes, measurement, or no, not a measurement? Now, there’s what’s called decoherence which we hear a lot in these kinds of discussions. Oh, when a large system interacts with a quantum system, there’s something called decoherence. Decoherence leads to a change in the quantum state of the system being observed. But decoherence doesn’t collapse anything. And it doesn’t single out one outcome. And you can actually prove that there’s no way to take the no-collapse axioms of quantum theory and get a definite outcome, to get one outcome singled out from all the others. So the question is, what do we do here? In principle, we need to provide a precise definition of what kinds of processes count as measurements and which ones don’t, and that’s called the measurement problem. We don’t have such a rigorous definition, and we haven’t for a hundred years. Maybe someone will come along someday and say, this is a measurement, and this is not a measurement, but at this point, we don’t have that statement. And without that, we just have this ambiguity. This is the famous measurement problem.

That’s one possibility. The next possibility is to say that there’s a definite outcome to the experiment, but somehow it happens without collapse. Or that the collapse is perspective-dependent. Wigner’s friend sees the collapse, but Wigner on the outside doesn’t. But if you take this horn of the dilemma, the trilemma, the quadrilemma, and you go down this path, what you’re saying is that Wigner on the outside has a quantum state that he uses to describe the system, and this quantum state is incomplete. And he doesn’t know the definite outcome that happened in the experiment. And if you accept that the wave function is incomplete, then you’re accepting that there are additional or hidden variables outside of the wave function, which, again, doesn’t agree with the textbook formulation of quantum theory, at least according to the usual axioms.

The third possibility is to replace measurement with something else, some kind of dynamical process whereby things collapse naturally when they’re large enough, and there’s a whole collection of approaches to quantum foundations called dynamical stochastic collapse approaches that do that. The wave function or quantum state is this thing, and for individual particles, it lasts for a very long time before it collapses, but when you put enough particles together, if any one of them collapses, everything collapses, and that explains why very large objects collapse very quickly. That’s option three. These approaches are very interesting. Some of them have been tested experimentally. They involve lots of new parameters, you know, and you have to guess the equations, and we don’t really have a well-motivated way of understanding which one should be used.

The fourth possibility is that there is no outcome. Nobody gets…

On a specific measurement. Either there is no outcome at all, which doesn’t really seem reasonable, or all outcomes happen in some sense, which is what Hugh Everett argued in his thesis when he looked at this problem. There is no collapse. We get rid of the collapse postulate, and Wigner’s friend splits into two parallel versions, each seeing a different outcome. One sees spin up. One can see spin down, and that’s inside the box. Also, Wigner on the outside doesn’t register a definite outcome. If the box is opened, Wigner sees the definite outcome, and Wigner splits into two versions, and the universe kind of unravels with the copies spreading and branching, or the wave function of the universe splits into multiple possibilities. That’s the many-worlds interpretation, and it’s a source of ongoing contention. There are many questions about how to understand fundamental questions about identity and the philosophy of mind in a world like that. It’s not clear how to understand probability in a world like that. There are ongoing debates on how to do that. That’s basically what we are left with. That’s what Wigner’s friend experiment left you with. You have to make a decision. I think there’s a fifth possibility, which is that quantum mechanics is wrong. And until we have a good reason to believe that this is just a mistake, maybe we should work under the assumption that we need to make the theory work better. I think that’s a good motivation to take seriously the fact that there’s something wrong with the postulates of quantum mechanics, and we need to do something about it. It’s not just ad hoc, but it’s either vague or incomplete or inconsistent in some way, and we need to fix it.

Where does that leave me, and where do I come in? One way to understand what I’m trying to do is to go back to 1922, when people were willing to give up the old pictures of particles and fields and actual things being there, and doing things, and behaving. Just as people were about to give up on finding the laws, and behaving based on those constituents that would give you an empirically adequate theory, perhaps it's premature to give up. Maybe we need to think more generally about what form the laws could take. By 1922, people already realized that it’s possible to have laws that are stochastic and probabilistic. Laws that didn’t tell you in a definite and deterministic way what would happen, but simply told you probabilistically what would happen. We knew about these. Einstein certainly knew about this kind of model. One of his great papers in 1905 was on Brownian motion. This helped provide strong evidence for the existence of atoms, it even allowed you to figure out the size of atoms, and this is a model that relies on what’s called a stochastic process. Stochastic from the Greek word stokhastikos, which means aimed at or guessed at. These are models where the laws predict probabilistic behavior. People knew about these things. Markov had introduced the Markov matrix in the early 1900s, 1906 or something like that. These ideas were in the air, and people were thinking, well, quantum mechanics is behaving in an unpredictable way. Perhaps the unpredictability can be absorbed into probabilistic laws, but people couldn’t come up with the right set of probabilistic laws at the time.

Over the years, people tried to return to this question. In fact, in Hugh Everett’s thesis, he talks about an effort by a physicist named Fritz Bopp who replaced the usual formalism of quantum mechanics with probabilistic laws, where particles move according to probabilistic laws. Everett actually said that this was a promising direction. He didn’t want to study it in his thesis, but this was something worth paying attention to. And many other people worked on these proposals, like Emery Fényes in the 1950s, and most famously Edward Nelson in the 1960s, but the models were extremely complicated. To get the same predictions that you get from the theory of quantum mechanics, you had to—this is the only political thing I’ll say—gerrymander the boundaries of the electoral districts. You had to manipulate—I mean this in the same metaphorical sense. You had to manipulate the laws. You had to start with what we knew was supposed to be predicted and then work backward and design these incredibly complicated stochastic laws in order to get the same predictions. I felt very strongly unmotivated. The spirit was a very interesting spirit. Let’s say the particles bounce around or the fields, whatever, evolve without a Hilbert space, without wave functions. Let’s just give them directly the probabilistic rules and in a way that makes those probabilities agree with the probabilities that we see for the outcomes of measurement in our experiments. But implementing the idea was hard. And it wasn’t clear that we had the right laws to do that. And the laws that make it work even in the simplest of cases require a lot of fine-tuning and manipulation. And when you look, for example, at Nelson’s stochastic mechanics, his laws are extremely complicated. And you actually have to find the wave function first from the Schrödinger equation, and then plug it into these laws, and the laws involve rates that move forward and backward—it’s incredibly complicated. And there were competing approaches. So, Louis de Broglie introduced pilot-wave theory in the 1920s that David Bohm independently discovered. And these theories were originally based on hidden variables, particles, that were somehow guided by the wave function. When you try to extend these models to beyond non-relativistic particle systems but, like relativistic systems and fields, it gets extremely complicated, and you end up deducing probabilities and stochastic laws again. So it seems there’s an understanding that ultimately, to describe most kinds of quantum systems, we’re going to need something like laws that are probabilistic in nature. But none of the probabilistic laws that people knew seemed to work very well. Either they didn’t work or they required so much fine-tuning that they seemed unmotivated.

In 2022, I was trying to teach a class, and I got a problem. We got to a point in the class where I was trying to present a little bit of quantum mechanics, and I was trying to decide how to do that. And I’d taught this class many times. Which class? It’s called Introduction to Theoretical Physics. It’s physics 19 here at Harvard University. And we cover a lot of topics in the foundations of theoretical physics in this class. During the semester, we present some concepts from quantum mechanics, but this is almost at the very end of the class, and I’m trying to give them a nice, self-contained preview of quantum mechanics. But these students are mostly very new to physics. Many of them have only just started learning college-level math. I can’t start talking about Hilbert spaces or C star algebras or GNS theory or any of that stuff. I need to find a more friendly way to get into quantum mechanics. And every year, I would tear up my syllabus and try again. I was never satisfied with the way I was doing it. Sometimes I’d say, maybe we’ll start with wave functions. Sometimes I’d say, maybe we should start with matrices. I couldn’t come up with the way I wanted to do it. And in the fall of 2022, I remembered some projects I worked on when I was an undergraduate and even in graduate school where I needed to use the theory of stochastic processes for, you know. In undergrad, it was like a special extra homework assignment for a linear algebra class. In my graduate studies, I actually used it in simulations. And, you know, the theory of Markov chains and stochastic processes, it bears some very vague similarities to quantum mechanics. It has probabilistic laws. You represent the probabilities of the system using a vector. The time evolution is represented by matrices. Now, they’re all slightly different. The probabilities don’t involve complex numbers. The matrices are what are called stochastic matrices. But there are these similarities. And so I sat down and said to myself, maybe I can find a way to bring these two theories together. The theory of stochastic processes on the one hand and the theory of quantum mechanics on the other hand. I had this kind of picture in my mind like walking on the edge of a cliff. On the one hand, you had kind of the classical world, classical stochastic processes, probabilistic processes. And on the other side, you had quantum mechanics. And there was this huge gap between them. And I was looking for a place where the cliffs got closer together and where it would be easier to jump from one to the other. So I started tweaking the mathematics of the theory of stochastic processes and quantum mechanics to try to bring these two pictures closer and closer together. I was doing this without knowing much about this prior work that had been done. The work of Bopp and Fényes and Nelson. If I had read all those papers, I would have given up because I would have decided it was impossible. But sometimes you can only do something if you don’t know that it’s impossible. I didn’t know it was impossible. So I brought these theories closer and closer together expecting that there would be a moment where I would need to make some kind of jump to go from one theory, classical physics, to the other theory, quantum physics. And the weirdest thing happened. The two cliff faces merged, and there was no longer a chasm anymore. For those who are familiar with second-order phase transitions in statistical mechanics, it was as if I had reached a second-order phase transition point. On the one hand, you have the two phases for some material like water, right? And then you go past a point, and the phases merge, and there’s only one phase left, right? There’s no longer a difference between liquid and gas anymore, right? And I didn’t understand what happened. I was baffled. So I started looking at the literature to see if there were other people who had done what I had done, and nobody had done what I had done. I think nobody thought it was worthwhile to try to do this thing that I was doing. But I started reading some of the literature on stochastic processes and Nelson’s work, and I was baffled. Why hadn’t they succeeded when this seemed to succeed? And that was around October of 2022. I know why. I went to the office of a colleague of mine, the great Logan McCarty, who runs science education here at Harvard University. He’s a quantum chemist. He knows quantum mechanics backwards and forwards. And I went and talked to him and went to him, and he had a whiteboard, and I started writing down everything I was doing. Because at that point I realized that the secret sauce, the thing that made this work, is that I implicitly gave up an assumption that I didn’t even know was supposed to be made or that anybody had made.

When people model stochastic processes, they usually assume that the process is a Markov process, named after Markov, like Markov, the Russian mathematician who introduced Markov matrices. A Markov process is a process where if you want to know what’s going to happen to your system, all you need to know is what’s happening right now. And then the laws tell you, at least probabilistically, what’s going to happen next. It’s memoryless. Memoryless, exactly. It’s memoryless. Now, Brownian motion is memoryless. That was Einstein’s stochastic model of atoms. And generalizations of Brownian motion, Wiener processes, and many familiar processes we’ve heard about, random walks, Poisson processes, these are all Markov processes. Nelson’s work, Markov process, they’re all memoryless processes. Now, I had a vague awareness that there was a subject called non-Markov stochastic processes. And I went and looked in textbooks to see if anybody had done what I was doing. And usually at the end of the textbook, the book says something like, and that’s all there is to the theory of stochastic processes. There’s a bigger subject called the theory of non-Markov stochastic processes, but this is extremely complicated. And it’s beyond the scope of this book. So it was very hard to find any books on non-Markov processes. And I looked in the literature and discovered that some people were trying to develop tools to deal with non-Markov processes. There are some that can be turned into Markov processes with a change of variables. They become what are called hidden Markov processes. And some, if they’re sufficiently non-Markovian, are extremely difficult to deal with. And so it seems like the wild west. There’s just this regime beyond the Markov approximation that not many people really felt comfortable exploring. It wasn’t clear how much we needed them. I mean Markov chains are really good at modeling the world around us. Do we really need to spend a lot of time worrying about non-Markov processes? And what I realized is that I had inadvertently introduced a kind of non-Markov process without realizing it. And as I dug into this more, I discovered that what I was working on wasn’t even a non-Markov process in the most common sense. It was a process of a more general type that first appeared in the research literature in 2006. And it’s called an indivisible process. Indivisible processes are in some ways the most general kinds of rules, dynamical laws, probabilistic generally, that fail to possess the Markov property. They’re more general than what we usually consider non-Markov processes. I think there’s more than one way to be non-Markovian, and people had a very specific idea of what they meant by non-Markovian. And this is like a more non-Markovian process than being non-Markovian. These processes first emerged when people were trying to understand the kinds of dynamical rules that describe Hilbert space objects, so not what was in my mind. They were already in the Hilbert space picture trying to describe how things in Hilbert space can evolve over time. And for them, the indivisible process was just a law that could tell you how your process would evolve from one time to another, but it couldn’t be broken down into smaller times. So you have a law that tells you how to go from zero to final time, but the theory doesn’t give you laws for intermediate times. That’s it. That’s indivisibility. Definitely not laws that can be iterated and composed into a larger law. And if you think about it, that’s certainly more general than divisible laws. I mean, because there’s a larger class where we don’t have some properties that we might like to have, which is divisibility. Newtonian mechanics is divisible. You evolve a system from an initial time to an intermediate time, and then you can stop, look at the state of the system now, and then, if you want, you can use Newton’s laws to tell you what’s going to happen next. Nice and divisible. Maxwell’s equations that describe the evolution of electromagnetic fields are beautifully divisible. And quantum mechanics, as it’s usually formulated in terms of the Schrödinger equation, is beautifully divisible. The Schrödinger equation tells you how the quantum state is going to evolve, and you can evolve it as far as you like, and then read off the quantum state somehow, and then predict where it’s going to go next. Is there another way to say indivisibility, or sorry, divisibility as a continuous time parameter? That’s a good question. Not necessarily. Because you can imagine an indivisible process where you are allowed to go from the initial time to any time you want, so time is continuous. But once you pick one of those times, you won’t have a law that tells you how to go from that time to later times. So time is not necessarily discrete here. It’s an excellent question. But what it lacks is the ability to divide time into smaller time intervals. Yes. So we don’t have much experience with this kind of law until this research paper came out titled Decomposition of Quantum Channels by Ignacio Cirac and Michael Wolf. Eventually, it got published in 2008 in the Journal of Communication and Mathematical Physics, and people are playing with it. You know, the paper gets a lot of citations. It wasn’t until 2021 until some people introduced the idea of a classical process that could be indivisible, like just a process where the system has some configuration, and it changes, and it behaves probabilistically, but the way it behaves probabilistically is indivisible. That appears in a figure on page 15 of this nice review article written by Mills and Modi, where they mention the idea that such a process is possible. I didn’t know it existed. I came up with the term indivisible independently because I didn’t want to use irreducible, which is used in many ways in physics. I didn’t want to use it, I was just looking for a word that would give you, and evoke the right idea. And it turns out they use the exact same terminology. So this was a year before I did it. And they didn’t connect it to quantum mechanics. They were just mentioning, oh, it’s possible, in principle, to have a classical process with probabilistic behavior, where the laws aren’t rich enough. They’re too sparse. They won’t tell you how to go from any time to any other time. You can only evolve from certain times to other times. The process as a whole doesn’t decompose. What I inadvertently did is discover these indivisible stochastic processes and showed that there’s just a change of mathematical representation that would take such a system with classically looking configurations, and it could be a system of particles, it could be a system of fields, whatever we’re trying to model. And if it has indivisible stochastic laws, then through this mathematical representation, this correspondence, you get the Hilbert space picture. That’s extremely interesting. Yes. And all the weirdness of the Hilbert space picture, superposition of states, and interference effects, are all related to what’s called coherence, phases, right? All these things are related to superposition, interference, and coherence. These things can now be understood. They can be given a meaning. So, in quantum theory, these things don’t have meanings. They’re just parts of the mathematics that show up in the formalism. We don’t understand what they mean. We know that interference shows up in experiments. We know that these things are called phases or coherence or superpositions, and we see these appearing everywhere. But they don’t have a direct physical meaning. We don’t know what they represent or what they really mean. What I was able to show is that by going from this concrete physical picture of things with definite existence, probabilistic evolution in this indivisible way, by going from that picture to a picture with the nice Schrödinger equation, where the evolution is divisible, it’s a nice differential equation, and very familiar mathematics, when you go to that seemingly divisible picture, you have to give something up. The indivisibility doesn’t disappear. It becomes all those weird phases and interference effects and coherence. That’s how they appear in this picture. It’s as if there’s no free lunch. If you want a mathematically convenient and simple picture, a picture where you can evolve the system for any amount of time you want, pause, and re-evolve, a nice divisible dynamical picture, you have to pay a price. And the price is that all this indivisibility becomes these weird phases. In a very loose sense, if you want to think approximately of these indivisible processes as being like processes with memory, all those interferences and phases, that’s the memory encoded in this weird way. And that’s the picture. That’s amazing. Yes. So in this picture, there’s no fundamental rule for the wave function. I mean you have things bouncing around according to laws that are more general than Newton’s laws of mechanics or even Markov process laws. And if you want to design a model of measurement, you simply design another system that interacts with your first system, and give the whole thing a simple nice set of indivisible stochastic dynamics. And what you’ll find is that the measuring apparatus will evolve stochastically into one of its final readout configurations with exactly the right probability that we expect from Born’s rule. But there’s no wave function. There’s no collapse. It’s just a big stochastic process happening.

So what influences the choice of time interval to define that time? You said you can pick any time interval, but then you can’t pick a subinterval. Right. That’s an excellent question. So, in order to start an indivisible stochastic process, you need a time that you can start at, right? And then the process tells you probabilistically where the system will end up at any smooth later choice of time. But is there only one special time that things can start at? The answer is no. What you can show is that when you have a system that interacts with a larger system, such that it’s not left alone, it interacts with a larger system, what you find is that if this larger system is exchanging information, with your original system, as would happen if you have a measuring apparatus that’s reading the configuration of your system, when it does that, you just let the stochastic process tell you what happens, right? There’s no postulate here. You just let it evolve. And what you discover is that you get a division event. The evolution splits at that moment, from the point of view of the smaller system. And now there’s a new time at which the evolution is interrupted, and you can now start again from that new time. So, in the indivisible stochastic process, it doesn’t mean that there are no times at which you can divide. There are times at which you can decompose the evolution, but the times aren’t given to you at the beginning. They’re not fundamental. They arise from the interactions between systems. Now, when you ask how this looks from the point of view of Hilbert space, in the standard picture of quantum mechanics, from the point of view of Hilbert space, what do those division events look like? They look exactly like decoherence. So decoherence, again, is when a large system interacts with a smaller system, the environment comes in, interacts with the system, exchanges information, and we get these changes in the quantum state of the other system. Changes that don’t pick out a single answer, but somehow change it. These changes wipe out the coherence effects. They kill interference effects. From the point of view of the indivisible stochastic process, it looks like the generation of a new moment, a new division event, an event where the evolution can be started again. And this happens at the end of measurements, and this is a physical process in this picture, and it coincides with what we might call measurement. But it’s generated by the laws from this picture themselves. Okay, I’m baffled. So, division events, are they just another renaming of what measurement is? Good question. So, in a way, if you give me a second system that I explicitly design. So let me phrase it differently. In the textbook postulates of quantum theory, measuring apparatuses are treated as being outside the formalism. They’re kind of in the background. They’re kind of like the person behind the curtain, right? We make a measurement, and we just change the quantum state of the system being measured, but we don’t usually put the measuring apparatus explicitly into the description itself. If you try to do that, if you try to put the measuring apparatus into the description, what you find after the measurement is that the measuring apparatus and its possible readout configurations become entangled with the possible configurations of the system being measured. There’s this entanglement that happens. And the question is, well, then what? Now they’re entangled, what do I do? What you’re supposed to do is collapse to one of those entangled branches. That’s the collapse. Now, if there’s a larger environment that comes in, and the larger environment looks at the measuring apparatus, it becomes entangled with the measuring apparatus. And there’s this process, this kind of formal process where you can kind of ignore the environment. And you see that there’s a suppression of all the interference effects, but there’s not yet a selection of a single outcome. You have to at some point pull out the fifth postulate and activate it and collapse to one thing at some point. In this indivisible picture, you can bring in this measuring apparatus, you can bring it in, and you let the whole system evolve stochastically. And what you find is that, by including it in the environment, the environment comes in, and you can treat them all as actual systems. None of them are outside the formalism, they’re all modeled as systems within the formalism. And what you find is that when you let these systems evolve, the system being measured and the measuring apparatus evolve probabilistically towards the configurations that we expect from the collapse postulate. And because of the environment, you can show that the measuring apparatus in the objective system undergoes a division event at the same moment of interaction that we call measurement. And this division event means that you can now restart the laws and start applying them from this moment. This division event is not assumed, it’s not a postulate. It’s something that just emerges from the usual rules of probability theory, when you actually model the systems explicitly. So, it’s a measurement in a way, but it doesn’t require a conscious being, it doesn’t require a human, and any system that has a very large number of moving parts, we call them degrees of freedom, in contact with a nice large environment with a large number of degrees of freedom, will work at a very high level of accuracy, and will produce measurement outcomes that probabilistically agree with what we expect from textbook quantum theory, and will generate a moment at which the stochastic dynamics of the universe. The system being measured can be restarted in the measuring apparatus and have its own dynamics. So yes, it’s a measurement in a way, but it allows us to get inside the measurement. It allows us to see the measurement actually happening, and it gives us a way to understand why measurement happens

Measurements with this thing, instead of just assuming they lead to the collapse of wave functions. Do you need to mention three different systems? Then the environment, the measuring apparatus, and the measurand? Yes, that's right. And we require that in standard quantum measurement theory anyway, in the usual textbook theory. And the environment as well? The environment is required, yes. Yes. Because the usual argument is that the environment is what generates the decoherence between the measuring apparatus and the system being measured. But when you say environment, I mean that in any realistic measuring apparatus, you have, you know, a part of the measuring apparatus that's actually directly involved in the measurement, but then you have the rest of the measuring apparatus. I mean, it's a big body, it's sitting in a box, it has dials, it has other experimental paraphernalia. I'm sorry, that means I will not be a good experimentalist again. But this, yes, there's always an environment approximately.

So, you first have to describe the environment, the measuring apparatus, and the measurand classically, and then you do some version of a quantum measurement procedure? Yes, so you can design them classically, in the sense that they have classically-like configurations. The measuring apparatus is some system, or physical body that can be in different possible configurations. Among those, at some level of coarse graining, we include, you know, the system is empty, it's not showing a reading yet, and the system shows an up spin on the dial, and the system shows a down spin on the dial, you know, but these are concrete configurations made of certain configurations of constituent atoms, whatever you want. And that means that their configurations, their ontology, their physical existence is conceived of in a roughly classical way. The quantum part is that the laws that they obey are not deterministic differential equations like Newton's laws, or Maxwell's laws of electromagnetism. And the laws that we give them are these non-separable stochastic laws, which we don't typically see in classical systems. The claim is that when you give these systems these more general kinds of laws, that's enough to make them behave like quantum systems.

So what's stochastic or random here? Good question. So all these systems have configurations, and these configurations evolve according to stochastic laws. That means that the configurations change over time in a probabilistic way. Now, why do they change in a probabilistic way? I don't know. These are axioms, right? At some point, you have a physical theory, you have to propose some axioms. I mean there is a position in how we think about fundamental questions in philosophy called foundationalism. You have to start somewhere. The standard textbook version of quantum theory, which works extremely well, and I certainly don't claim that any of the predictions it makes are wrong. They're all spot on. These axioms are axioms, and they're very strange. We assume Hilbert spaces and state vectors or density matrices and the Schrödinger equation and unitary operators and all the collapse postulates, and we assume them because using those axioms we get empirically correct predictions. This picture that I'm presenting also has axioms. The axioms are simpler. Every system has a set of possible configurations, classical configurations, and the laws are these non-separable stochastic laws. That's it. Now, if you want to ask why the laws are non-separable stochastic, I don't know. The best I can say is that when we try to explain things, there's an argument that you want to start with the least restrictive probabilities, the most general probabilities, and then argue that under certain circumstances you get more restrictive probabilities. Non-separable stochastic laws are the most general kind of laws you can imagine. Maybe you can go beyond that. I don't want to say they're definitely the most general, but they're certainly more general than Markov laws. They're more general than the usual way we think about non-Markovian laws. They're more general than deterministic laws, laws of the differential equation type. To me, it makes sense that you would start with the most general type of law, and then explain why under special circumstances we expect to see Markovian processes, and why under certain circumstances we expect to see deterministic second-order differential equation type of laws. That's the task of understanding the classical-quantum transition, of understanding the so-called classical limit. How do we go from the laws of quantum mechanics to laws that look like the laws of Newtonian mechanics or look like classical Markovian processes? These arguments also persist in this picture. We don't start with deterministic laws. We don't start with Markov chains or anything like that. We start with these more general laws, and then we adapt the usual arguments that people usually use in trying to do that to arrive at the classical limit, the classical regime, we see why under certain circumstances we get large systems that have a classical appearance, whose laws appear deterministic or at least appear like probabilistic Markovian laws. To me, that's the right direction for interpretation. You start with the most general, least restrictive types of laws, and then you move to more restrictive laws.

I think I'll say that I don't feel the need to explain why particles or things behave probabilistically. Probabilistic behavior is the default behavior. That's what happens if we don't have a good reason to believe that laws should be deterministic. And then the goal is to clarify why under some circumstances we get predictable, deterministic behavior. But the truth is, I don't have a deeper explanation for why we expect particles to wiggle probabilistically. That's what it means to have a non-separable stochastic law or a stochastic law in general. And your probabilities, are they classical probabilities or are they quantum amplitudes? That's a great question. They're classical probabilities, and that's kind of key. So this basic picture, the claim is that every quantum system at its beating heart is really a system with configurations. If it's a particle, we're talking about where the particle can be. If it's a field, we're talking about the different intensities that the field can have in space. If it's a system of abstract bits, like in a computer, it's a configuration of zeros and ones. That's what I mean when I say classically-like ontology or classically-like configuration space. And the probabilistic laws just specify ordinary probabilities for these things. They say the system starts in some configuration, and then the non-separable stochastic law takes the form of laws that tell you what the probabilities are of where it will end up. And these are just ordinary probabilities. They're real numbers between zero and one. They sum to one. There are no complex numbers at all in this picture at this stage. But if you want to describe the same process in a mathematically more convenient way, I mean non-separable processes are actually rather inconvenient to work with, especially if your system is complicated, because we don't have this ability to factorize the laws. It's very hard to use them directly. So instead, we use this mathematical change of representation into the Hilbert space picture. We translate all the components in this underlying process into the Hilbert space picture. In the Hilbert space picture, we have nice differential equation type laws, the Schrödinger equation. We have complex numbers appearing. And then the probabilities that you have in the non-separable stochastic theory become what we call the squared moduli of the complex numbers. The complex numbers whose squared modulus you take to get probabilities are called amplitudes. So only when you go to the Hilbert space picture do you need amplitudes. And amplitudes do all sorts of funny things. They interfere with each other. If you want to write down a process where the system can go one way or another, you're supposed to consider all the probabilities and then sum all the amplitudes together, and there's interference, right? But all of that happens in this kind of mathematical representation, this Hilbert space side of things. And the claim is that this doesn't tell you what's actually happening in the system. This is just extremely convenient mathematics. And again, it gives all the right predictions, right? The idea here is that you recover the usual axioms of quantum theory, but now you understand where they came from. And now they're no longer inconsistent anymore, because at the fundamental level, there's no special role for the observer. There's no special role for measurements. All the processes that we think about in quantum mechanics are just ordinary processes. They just look weird and wacky when you look at them on the Hilbert space side.

Have you encountered any resistance from people who like their quantum mechanics mysterious? Yes. Explain. Well, I mean quantum mechanics is very exciting. Actually, let me rephrase that. That doesn't mean that they like their quantum mechanics mystical. Have you encountered any resistance from people who like their quantum mechanics mysterious? Puzzling. Yes, they like it to be shaken not stirred, right? I mean a little. Yes, I think there's a sense that this picture is deflationary. It deflates. It takes away weird pronouncements. Oh, the cat is alive and dead. And it deflates them and says, actually, it's really one or the other. And we just represent it mathematically with a state that looks like a superposition of alive and dead because it gives us a factorizable evolution that we can use. It's easier to study mathematically, but that's not reality. It deflates that. Yes. Just to clarify, deflationary is a philosopher's term for saying there's nothing to see here, folks. Just calm down. Yes, it's basically like sucking the air out of a balloon, right? It brings it down to earth, and that's basically what this does. I think it makes quantum mechanics, in some ways, more boring. And you know, while I was working on this, I kind of felt that when young people come into physics and start learning about quantum mechanics, it's incredibly exciting to think that you might be able to understand what happens when Schrödinger's cat is alive and dead. And you know, particles can go through both slits in the double-slit experiment simultaneously and interfere with themselves. And you know, everything is entangled with everything else. And maybe when we think about things, we humans are special. We collapse things. That's boring quantum mechanics. You're saying that at some level, quantum systems, if things are done right, might just be kind of mundane. They just don't have a Hilbert space wave function. There are just things, atoms, chairs, whatever. And they just evolve in a non-deterministic way, a highly non-deterministic way. Well, still probabilistic. We can still assign probabilities to things because you can imagine laws that are so non-deterministic that we can't even assign probabilities to them. But we can assign probabilities to them. But then a lot of the magic has disappeared, right? There's no special role for humans. In a sense. Means what? The history of science has been pushing humans further and further from the center of the story. You know, at first we were the center of the universe. Then we weren't. Then maybe we were the center of all observations and quantum mechanics. And this takes us right out of that. There's nothing particularly special about human observation or the human mind. And I think that with evolution, we might not soon be the smartest people on this planet, the smartest beings or entities on this planet. Um, so I don't know. I mean I can only speak for myself. I felt a little sad when I thought there's a way to think about quantum mechanics where there's no wave function. Nothing branches into multiple realities.

So you mean there's no fundamental emergent wave function? Well, good. It emerged. Yes. So wave functions play their role, so there's a way to take Newtonian physics and introduce kinetic energy, which is energy associated with things moving and potential energy, which is energy associated with the configurations of things. And you can take these two quantities. They are more abstract than being like particles and forces and things, but you can imagine kinetic energy and potential energy. And you subtract them from each other, which is kind of weird. Usually we think that the total energy is the sum of these things. You subtract them and then you can integrate this thing over time. And you get a mathematical construct called the action, the action functional. We actually cover this in my class now. The thing that you integrate kinetic energy minus potential energy is called the Lagrangian. At least in the simplest cases, it's kinetic minus potential energy. And you can use this to encode the laws of your system. You can use this to describe how the system behaves. But you can ask what is the action? Like, is it a physical object? You can make the action evolve in time. You get what's called the principal Hamilton function. It obeys a partial differential equation. And I didn't mention it, but this is the beginning of Hamilton-Jacobi theory. And from studying the action evolving in time, and the principal Hamilton function, and its differential equation, Schrödinger was led to discover the Schrödinger equation. It was a beautiful guess of the partial differential equation obeyed by this action. So this action is a function of time or principal, the principal Hamilton function is very much like the wave function in quantum mechanics. So much so, as I said, that it inspired Schrödinger to develop the wave function. In a sense, Schrödinger's wave function was an exponentiation of the principal Hamilton function. And now you can ask, what's the meaning of the principal Hamilton function? Is there a physical body in the world that evolves according to this partial differential equation, the Hamilton-Jacobi equation, the equation satisfied by this principal function, this integrated kinetic minus potential energy? I think most physicists would say no. They would say it's just a mathematical convenience. It's a construct of the theory. We can use it to generate predictions. We can use it to make predictions. And that's great. It's beautiful mathematics. But there's nothing in physical reality that it describes. And what I'm saying is that the wave function is basically like that. It belongs to the same conceptual or metaphysical category that the Hamilton's principle operates in. Sure, it's in the mathematics. It's convenient. But there's nothing physical that it describes. And because there's nothing physical, there's nothing branching. There's nothing splitting into branches. So you would never think about the possibility of many worlds or anything like that. There's nothing in the ontology of the picture as I've presented it that would play the role of branching realities. So things like many worlds don't even get off the ground in this picture. So I think that's what I mean when I say there's no wave function. I mean that in a somewhat precise sense. I don't mean that we shouldn't use wave functions in mathematics or use them to make predictions. But we should be a little bit more humble in the way we talk about them.

Have you spoken with Sean Carroll about this? That's an interesting question. Not yet. But at some point we will hopefully talk. Yes. It's an interesting question because I haven't traveled much in the last year or so. So our paths haven't really crossed. But I would like to have a conversation about this at some point. Any Everettians or many-worlds? People? A little. A little. But I think you talked about the mysticism. You talked about how people like their physics. The truth is people get good at doing physics in a certain way. And we develop intuitions. When you learn to play a musical instrument, you learn to play the piano, you learn to play chess, you learn gourmet cooking. And you do this over many years. And if you have the right mix of motivation and talent and skill and the right environment and the right mentors, the right mix of all those factors, you will end up becoming very good at it. And you can do amazing things. You can play beautiful musical pieces. You can prepare a fantastic cuisine. You can become a grandmaster in chess. And if somebody comes along and says, I think you should do it differently. You should go back to the starting board. You should sit at the piano bench like this. And you should hold your hands like this. Or you should play chess. And you should start the pieces in a different position. And we should change all the rules. Chess pieces should move differently. You're going to set the person back. And you can imagine why somebody might not want to do that. And not for any negative reason. It's just that they've honed themselves to do something really well. And asking them to go back to the drawing board and start again is a big ask. And so I would basically say that anybody who's been doing quantum mechanics in a certain way for their whole careers, who have developed all sorts of intuitions, have a good sense of what to do with wave functions and things. You know, I can imagine somebody, in this situation, being hesitant to go back and rethink all the fundamentals from scratch. Especially if what they've been hearing for most of their careers is that the fundamentals are fine. Quantum mechanics is just a tool to generate predictions. Why should I go back and think about any of these things? So I don't think I'm still not doing that, I'm not criticizing any of that position. But what I've definitely found is that newer people in physics, students, tend to be more interested in talking about a project like this. But there are definite exceptions among professors as well. I mean, I have, you know, some wonderful colleagues who have been incredibly receptive and open-minded, and we've talked about this project a lot. And I've gotten a lot of really good input from people. So, you know, but I think it's a mix of intuition-building and also what draws us to studying quantum mechanics. I mean quantum mechanics is a weird theory. I mean the belief that you're going to spend your time imagining the world is a wave function or a state vector or some abstract object living in a high-dimensional configuration space or in somewhat modern parlance in a high-dimensional Hilbert space. Hilbert spaces are beautiful. The mathematics is incredibly elegant and has a certain kind of spice to it. But you get complex numbers floating around. We don't know why, you know, at least in the way we usually talk about it. I can imagine being romantic about that. I was. I feel like that's what drew me to quantum mechanics. So I can definitely see why some people want to hold onto that. And I think that of all the people who take a romantic view of quantum mechanics, I don't think anybody takes a more romantic view than Everettians, people who work on many-worlds interpretation because their picture of the world isn't just weird. It's deep, right? The idea that every time a quantum event happens, the universe splits into two branches or three branches, but in a more sophisticated and more precise version of Everettian approach, it's a limitless profusion of branches. I mean that's breathtaking and amazing and forces you to confront some very deep questions in metaphysics, in the philosophy of mind. It's just very rich with inquiry. I can imagine people who work in this area wanting to hold onto it. So I'll give you two examples because I can relate to this, being stubborn and bullheaded. So when I was learning about the quantization of quantum mechanics and reading papers like, okay, here's another way you can think about quantum theory as a dagger-symmetric monoidal category. And I look at it and think, okay, but so what? I still have that, so what? But I think part of it is that I'm used to doing quantum mechanics in an alternative way. Now, my dad has set up his office such that if you move one piece of paper, he will be upset with you, even if it's out of the way, he keeps everything in his mind. I had to install Windows XP two years ago, Windows XP on his computer because he's familiar with Windows XP and he wants to use DOS and he can't run QBasic on Windows 10 without some emulator or virtual machine. So my dad is kind of like, what you classify as, or what you describe as some people who are used to quantum mechanics the old way, at least when it comes to just not hating it for being deflationary, but hating it for being different. Now the question could be, well Jacob, what does this give me? What does your approach give? Why should I learn it? You just said it looks like ordinary quantum mechanics. Okay, great. Maybe for you it's a more beautiful version, and maybe for somebody else more complicated, but whatever, what does it give you? What's the payoff? That's a great question. So the first payoff is a correction of the axioms. The axioms had this ambiguity or inconsistency or incompleteness to them. They had this puzzle about what measurement is that seems to play a role in the axioms, but it's not defined by the axioms. And you'll encounter problematic thought experiments like the Wigner's friend experiment. And the Wigner's friend thought experiment is that it's not the same as the black hole information loss problem by any means, but there's some similarity. There's this big problem in quantum gravity. Black holes seem to radiate according to work going back to Stephen Hawking. And there's this tension between the evaporation of black holes quantum mechanics and unitary evolution or Schrödinger evolution, which is information preserving in some sense in quantum mechanics. And the fact that black holes supposedly can't let information escape. There's this tension between all of these things. That's the kind of tension that we see in thought experiments like the Wigner's friend thought experiment. And it's not as if the black hole evaporation problem is easily accessible experimentally. I mean, to go out and do experiments on black hole evaporation, you would have to prepare 10,000 similarly prepared black holes, which we don't have to do. Wait for many times the age of the universe for all of them to evaporate, and make sure they all evaporate inside a perfectly sealed room with ideal detection mechanisms to collect all the Hawking radiation that comes out with sufficient sensitivity that we can detect, you know, incredibly tiny entanglement effects, right? Then we subject all the data to a level of analysis that we believe might be computationally impractical. I mean this is not a practical problem, at least as far as we know so far that could change. But as far as we know, it's not a practical problem. Yet, this is a serious area of research. Whereas the inconsistencies in quantum mechanics are not considered serious for some reason. I don't quite understand. There's some sociology there. But what I would say to quantum physicists is that one of the payoffs of this approach is that it resolves some of those inconsistencies, which might in itself be a good thing. It does so in a way that doesn't require that we only model one kind of quantum system. One of the drawbacks of pilot-wave theory like Bohmian mechanics is that it works really well with very simple kinds of systems consisting of a fixed number of non-relativistic particles, but it has resisted precise generalization beyond that. So it's good to have a more independent and simpler model than the Bohmian approach. And it doesn't face many of the puzzling metaphysical riddles that one has to grapple with with Everettian quantum mechanics. You know, we don't make millions of copies of ourselves in this picture. So it provides, I think, one way and maybe at this stage, maybe the only attractive way, in my view, of resolving these inconsistencies with quantum mechanics, although mileage may vary and I don't want to speak for everybody by any means that extends. That to me is already something. And the second thing is that it provides a picture. Quantum mechanics, when I teach quantum mechanics to students, they expect me to talk about waves moving in three-dimensional space. And when I show them the axioms, many of them are quite disappointed. They're like, but where's the picture? Isn't it as if there's an electron and it emits a photon? And I say, well, the textbook axioms don't really talk about an electron here emitting a photon there. According to the axioms, this is just added for color. We draw pictures of electrons emitting photons only to help tell the story. But none of that is actually sanctioned by the axioms of quantum mechanics. There's also the problem that there are a lot of phenomena that occur in the world that are not measurements. You know, when early gases mix in the early universe, when distant stars collide, when a piece of aluminum is sitting on the surface of Mars and doesn't collapse under its own weight, when birds forage, when humans fall in love. These don't seem to be measurements. Yet the axioms of quantum theory, the textbook theory only makes predictions about measurement outcomes, probabilities of measurement outcomes, averages of measurement outcomes weighted by probabilities of measurement outcomes. And that's it. That's a very narrow class of phenomena. How do we go from that narrow class of phenomena to the broader class of all the other things that we think happen in nature? Or are we just going to say that our talk about all those other things, about birds foraging and early gas mixing, is just for color, just storytelling? And this allows quantum mechanics to be a theory of a much larger class of phenomena, not just a narrow class of measurements. I call this the class problem in textbook quantum theory. But ultimately, I think physicists will be interested in applications, practical applications. So – or you were hoping. Well, we were hoping, but yes. But in my experience, physicists, you know, want to know, what can we

What should we do with this? I think there are some things you can do with this. The first is that indecomposable random processes, at least classical indecomposable random processes, are currently three years old. This is a completely unexplored area of dynamical systems theory. It’s like opening a completely filled sketchbook, turning the page, and finding a completely blank new page. How often does that happen in mathematics or science, that you turn the page and open the door to a completely new area of research? I don’t know much about the mathematics of indecomposable random processes because nobody knows it yet. I think that’s interesting. I think it would be very exciting for people to think about these kinds of systems and discover their properties.

Perhaps the best way to study an indecomposable random system is to write down its Hilbert space and essentially do quantum mechanics. Just as sometimes the easiest way to study a Newtonian system is to write down its Lagrangian and do Lagrangian mechanics. Maybe we can understand things about these systems according to their own merits. Maybe when you don’t start from Hilbert space, but you start with these different axioms, you can point to new ways of generalizing quantum mechanics that were impossible to imagine before. If you start with Hilbert space, that limits your imagination about what you can do as well. If you don’t start with Hilbert space, you’ll start with a much more general kind of probabilistic system as I was talking about in this picture, this indecomposable random process. Maybe there are deformations you can make to the assumptions, changes you can make to generalize these systems in ways that don’t give you Hilbert space. You can generalize these systems and get something else that doesn’t give you Hilbert space that you never could have gotten to if you had started from Hilbert spaces. And perhaps we need such a generalization in order to make progress on fundamental questions related to quantum gravity. So I think that’s another potential area of application.

And there’s another area, which is that if there is this correspondence between Hilbert space representations of quantum systems and classically looking probabilistic systems, especially beyond the Markov approximation, this might point to efficient ways of simulating classical non-Markovian random systems using quantum computers. So one of the open questions is if we had a quantum computer today, what would we use it for? There are already a number of known algorithms. There’s no proof that the algorithms are stronger than what you can do classically, but it’s widely believed that many things you can do with these quantum algorithms are much more efficient than what you can do with classical algorithms. But it’s not as if every question you would want to do on a classical computer can be done more efficiently or faster on a quantum computer. So knowing what kinds of things a quantum computer will help us do more efficiently is a really open question. And when you establish a correspondence between quantum systems and a large class of random systems, you open the door that there are some problems that might be difficult to simulate classically that are easier to simulate quantum mechanically. I don’t quite know what those are yet. And that’s the area of research that I’m working on now. But I’m talking to people who work in quantum simulation theory of classical random processes, and we’re trying to identify perhaps there are ways to use these correspondences to provide more efficient ways of simulating these systems and provide a novel application for quantum computers.

Now, there’s also the possibility that under some circumstances, indecomposable random systems might behave differently than quantum systems, at least textbook quantum systems. Perhaps there are slightly different predictions regarding the Wigner’s friend thought experiment. Unfortunately, performing a Wigner’s friend thought experiment is probably beyond our experimental capabilities for the next thousand years. Who knows? But there are some instances where you want to model a quantum system, and you need to make additional assumptions in order to make actual predictions, assumptions that go beyond the textbook axioms. For example, if you want to design a very complicated quantum system, you might have to add some extra axioms about, you know, what are typical initial states of the system, and what kind of typical evolution we expect. These go beyond the textbook axioms. And it’s possible that when you start with different axioms, you’re actually modifying some of those assumptions. And therefore for some classes of very complicated quantum system, it’s possible to get slightly different predictions. But these are all very speculative claims. There’s a lot of work that needs to be done to determine if any of these things can work and when. So I want to be very clear, we’re just at the beginning here. But even the existence of these correspondences at all, I think it opens a lot of doors. And there’s nothing more exciting to me in science or philosophy than open doors. Okay, this is certainly cutting edge, like the very latest developed in the last two years. And also, it may be widely applied soon and may already be on the right track.

Does it offer any insight into QFT, which relies on Fock space, which relies on Hilbert spaces? Yes, so that’s an excellent question. So quantum field theories fall within the axioms of textbook quantum mechanics. Those axioms that I mentioned, and Hilbert spaces and all that stuff. A quantum field is a particular kind of system that lives within the framework defined by those axioms. What makes quantum fields, well, there are many things that make them complicated. The one thing that makes them complicated is that they have an infinite number of moving parts. Unlike a system that consists of five particles, they have an infinite number of moving parts. We see that these systems have infinitely many degrees of freedom. And this causes all kinds of mathematical complications. When working with quantum field theory, as if ordinary physicists are using quantum field theories today, you often have to regularize the theory. I mentioned Planck introducing the Planck constant for regularization. Yes. So the regulators that we use temporarily make many of these infinities finite. So that we can do the calculation. And then what we do is we try to send the regulator to infinity. And usually we can do that. Then it goes away. And we believe that we have a good and strong prediction that doesn’t depend on the regulator. Unlike the Planck constant, the regulator goes away and we believe we have a strong result that holds in the infinite connected limit. And there’s a beautiful and complicated story called renormalization and effective field theory that’s all about how to do this in a mathematically consistent way, and in a reasonably consistent way, and generate good predictions. I wouldn’t have time to go into all of that. But ultimately, yes, this is still just a quantum system, a very complicated quantum system. And insofar as you can generate the axioms for any quantum system, you can generate them for quantum fields starting from an indecomposable random process. According to this picture, a quantum field is a collection of non-localized entities, spread out in space with configurations. And those configurations evolve in this indecomposable random way that can be modeled in the language of Hilbert space. In a sense, it’s like any other kind of system, you’ll still face the same mathematical complications, and maybe new complications because of the fact that we have an infinite number of complications and you have to do mathematical tricks to make these things finite. But yes, the answer is that quantum fields fit into this framework. But it will be inherently very complicated because quantum fields are extremely complicated.

And do you reproduce the precise assumptions of quantum mechanics? Or do you have some deviations that there’s an experimental way to test whether the random approach is the more correct approach? That’s a good question. So at the level of the correspondence, there’s just a map that takes you from the components of the indecomposable random theory to the corresponding Hilbert space description of the quantum Hilbert space representation. There are principle deviations because collapse doesn’t fundamentally happen in this picture. However, the deviations will be roughly similar to any no-collapse formulation of quantum mechanics. So Bohmian mechanics, the pilot-wave theory of de Broglie and Bohm, where particles are guided around by wave functions, ready always in quantum mechanics, you know, the many-worlds interpretation, these are all non-collapse interpretations. Collapse doesn’t fundamentally happen. There’s no axiom of collapse. And so you will get small discrepancies in predictions of these theories compared to textbook quantum theory. And I can make this very clear. Go back to Wigner’s friend. According to textbook quantum theory, Wigner’s friend inside performs a measurement that causes the quantum system inside the box to collapse. That system collapsed. It collapsed for everybody. It collapsed for Wigner’s friend in the box. It collapsed for Wigner outside the box. It just collapsed. In a no-collapse approach, like many worlds or Bohmian mechanics or my approach, there is no actual collapse that happened. And what that means is that the wave function that Wigner uses on the outside, the quantum state is an uncollapsed quantum state. It contains super-position interference phase effects that have not been erased. They haven’t been erased because there wasn’t a collapse. So, in principle, Wigner on the outside, using a sufficiently powerful measuring apparatus, could detect those small discrepancies, which might be inconsistent with textbook quantum mechanics. But detecting them would be extremely difficult. So there’s a paper from a few years ago, Scott Aaronson and Lenny Susskind and I and I believe one other author where we showed what people call quantum necromancy, which just means that if Wigner could design an experimental apparatus that was sophisticated and sensitive enough to detect those small discrepancies between a no-collapse interpretation and textbook quantum theory, then that apparatus with some relatively minor modifications could take a long-dead cat and bring it back to life, which we usually consider a thermodynamically impossible feat, right? That’s what they call necromancy, like bringing the dead back to life in a magical way. It would be more like unscrambling an egg, right? Like taking a scrambled egg and getting it back to the original egg. I mean that really goes beyond any technological capability we can imagine. So, yes, it does make slightly different predictions, but only insofar as other no-collapse interpretations make the same slightly discrepant predictions. And unfortunately, these discrepant predictions are so tiny that they’re extremely difficult to measure. Perhaps measuring them is easier than measuring the black hole information problem. But sort of on the same level of extremely small suppressed effects.

Okay, before we end, I want to know what a particle is from your point of view? And also is your view called an interpretation? Is it a theory? Is it a correspondence? What do you call it? Why don’t we just settle that now? I don’t have a good answer for that question. I think you can call it whatever you want. One can think of it as a physical theory. It has variables and components in it, and those components have laws. So it’s a physical theory. It’s a physical theory that makes the same predictions as quantum theory in all realistic scenarios to the best of my knowledge. You can think of it as an interpretation of quantum theory, in the sense that if you start with the formalism of quantum mechanics and you ask, how do we interpret things? What do things mean? This provides a platform to actually talk about what’s actually out there in nature. I should say that it’s not a one-to-one picture. A given quantum system may represent many different indecomposable random processes just like a Lagrangian or better yet, a Hamiltonian formulation of classical physics can describe many different Newtonian systems. There’s no one-to-one correspondence between these pictures. You can think of it as a formalism because it’s a new mathematical framework. I think what you can’t call it is a recipe or a prescription because it’s more than just a list of steps to follow. It’s more than just an algorithm, just as the axioms of quantum theory are just an algorithm to follow. It actually tells a story about the things that happen. I don’t know if that answers your question. Do you want to follow up on that or do you want me to talk about particles? What is a particle? Classically, of course, a particle is a point-like entity that is characterized by a mass of some kind. The mass can be zero. I think that’s a tachyon. It can have some strange imaginary mass. It is characterized by spatial degrees of freedom, meaning that it has a location in some conception of physical space, usually X, Y, Z coordinates when I think of it that way. It will generally have different locations, different times, which we can think of as like a trajectory. If it’s a truly point-like particle, it doesn’t have internal structure. It doesn’t have size. However, it can carry certain other quantities. It can have other attributes like electric charge, which is related to how it interacts with fields and other kinds of particles. It can have an intrinsic form of angular momentum. You can have kinetic energy, but kinetic energy can sort of freeze out and become a kind of rest energy, like the internal energy of the particle. Angular momentum is sort of orbital momentum, it’s sort of spinning momentum, but you can also have frozen orbital momentum. And that’s called intrinsic spin. Particles can carry this intrinsic spin which can be converted into other forms of angular momentum, just as rest mass can be converted into other forms of energy. That’s what a particle is. It’s a thing with this collection of possible properties.

In quantum mechanics, at least since Wigner’s work on representation theory, particles have been understood as the simplest kinds of quantum systems that have well-defined nature or behavior under spacetime transformations. If you ask what’s the simplest kind of quantum system whose Hilbert space is as small as possible, it doesn’t include anything weird, but when we move through space or we rotate or we change the inertial reference frame, we change the frame of motion, we just get a new element from the same Hilbert space. We don’t get something new, but it’s at its minimal. It’s irreducible. Then you have a problem in mathematics. What are the irreducible unitary representations, which just means they’re represented by certain kinds of operators in Hilbert space, representation of the spacetime symmetry group, the Poincaré group, the abstract representation of moving through space, changing frame of motion, rotation. That’s a problem in mathematics. You can classify them, and what you find is that there are distinct kinds of Hilbert spaces with these properties, characterized by mass and spin and charges. They have kinematic degrees of freedom and so on. That’s usually how we think about particles in quantum mechanics. That’s all we can say in quantum mechanics because all we have is Hilbert spaces in the textbook formulation. If you’re going to describe a system of particles, you have a Hilbert space. All we can do is talk about what the Hilbert space is, and what its structure is, and the structure of the particle is tied to these transformations in spacetime. In the picture that I propose, particles go back to the classical picture. A particle is a point-like object in space, in three-dimensional physical space if we believe physical space is three-dimensional. With different properties or dispositions that interact with other particles. Dispositions? Yes. That means that when you interact with a particle, it might be disposed to give you certain outcomes on your measuring apparatus, that’s all I mean. If you measure its position, you’ll read what its position is, but if you measure some more complicated feature, what you get on your measuring apparatus will be a more complicated outcome of the interaction with the particle. It has a disposition to produce certain kinds of behavior. In metaphysical talk, we can say that some properties are categorical. They’re explicitly there as they are, and others are dispositional. Meaning that you look at it and what you see is actually like how the particle is disposed to show you what it will do. But ultimately, particles are pretty much what they are in the classical picture. Just point-like entities that have certain properties, different positions across time. So that’s all a particle is according to this picture. And I should say that I’m not saying particles are the fundamental ontology of nature. I don’t know what the fundamental ontology is. But so far in your theory, what’s the fundamental ontology? We go back to the old classical way of doing things, which is that if you want to model a particular system, you propose a certain ontology for it. You have to say what you think the configurations are. If you want to model particles, that would be arrangements of particles. If you want to model fields, it would be non-local field intensities distributed across space. If you want to model strings, if you want to model bits on a computer register, you choose the right ontology for the right kind of system and then you model it. It’s possible that there’s some truly fundamental ontology of all of nature. I don’t know what that is. I think for some people, especially people working in Everett quantum theory, the idea that the fundamental ontology is state vectors or wave functions in Hilbert space is a very appealing idea. Meaning that we already know what the fundamental substrate of all of reality is. Hilbert space, there’s some Hilbert space and there’s an object or objects in Hilbert space and that’s the fundamental ontology of everything. I’m not saying that. Hilbert spaces aren’t fundamental in this picture. We go back to needing to be somewhat metaphysically modest. We know about some systems. Some systems look like particles, some look like fields. Maybe particles are emergent from fields and we don’t need particles as a separate category. Maybe fields are fundamental, and maybe they’re not. I think it’s premature to try to guess what the fundamental ontology is right now, and in this picture you don’t have to. All you have to do is choose the ontology you need for the model you want to talk about, particles, fields, registers on a computer, and then proceed from there. Maybe someday we’ll discover that there’s one ontology that’s one ontology from which we can get all the other models. That would be a very exciting day, but I don’t think we can guess what it is today and I don’t think we can just assume it’s Hilbert spaces and be done with it. I think that’s premature.

There’s a lot more I want to talk to you about. And we haven’t talked about causality either. There’s a whole other connection with causality. Reichenbach’s common cause. Right, we haven’t talked about Reichenbach’s common cause principle. And that’s another reason why I think this work is likely to be interesting because it has some connections with causality that I’m very excited about. I think we should explore that next time. Yes, maybe we should talk again. Professor, it was great to see you in person. I’ve been following you, again, I politely follow you. Diplomatically, whatever. You understand. For some time now. Why don’t we end, I know you’re about to run off to see your students, why don’t we end with your advice to students? Oh. Actually, two. Your advice for the students watching and the researchers watching. I’ll just use my backup advice, which I think has been really helpful for me. People have told me this and maybe I’ll share it. We all come to the work that we do with a distinct set of strengths and weaknesses, as I just said. You know, everybody is a certain collection of strengths and weaknesses we’re like a point in a very high-dimensional configuration space or parameter space. So we’re not ranked. You can’t order people. We’re not an ordered set. It’s very easy for students who are studying science to think that they’re operating under a linear ranking system. There are people better or smarter than them. And if they’re not as smart as those people, they won’t be able to make contributions, they won’t be able to do good work. And in my experience, having worked with many students over the years, but also having read a lot of history and the history of science, we find that it’s really true that all of us, all of us are points in some very high-dimensional parameter space. We’re not ranked. Knowing how to leverage your unique profile of strengths and weaknesses to make progress in the work that you find meaningful, knowing how to do that is part of what it means to enter the academic workforce and grow and learn. And develop. So I would say don’t worry so much about where you think you fall in the ranking and focus more on figuring out how to bring your unique profile to the table and how to use it to make progress in things you find meaningful. That’s probably my best advice.

And for the researchers watching, not advice, but any message you want them to see, well, you can offer advice too. Interdisciplinarity is great. We should have more focus and spend more money on places where disciplines intersect. I work at the intersection of physics and mathematics and philosophy and now, you know, statistics, you know, dynamics. There are many different fields that I meet at the place where I’m currently working. And I think this isn’t new, but I think some of the most exciting work that happens in scholarship generally and certainly in science or philosophy specifically happens at interdisciplinary boundaries. So I would say that maybe not to researchers, but to the people who fund researchers. Yes, spending more time thinking about how to foster work at disciplinary boundaries where there’s a genuine cross-pollination between disciplines, I think that would pay huge dividends for progress in all our work. I agree. Last night, one of the reasons I’m here besides seeing you was at MIT, there was an event by Adi here yesterday, last night, called “ekkolapto,” which was about neuroscience, cognition, biology, physics, and language. Yes, it was a mixture of all kinds of major fields in science and technology and engineering and math (STEM) and the arts where they intersect and can be used to create something really useful. But you have to be rigorous about it. You can’t just, you know. So that was fun. Anyway, thank you. It’s been a pleasure and I know you’re keen to move on and I appreciate that your students have been waiting for that. That’s great. I appreciate your time. Thanks for having me. That’s great. I really appreciate the invitation and all the work you’ve done on this fantastic podcast series. It’s great work talking to audiences and talking to people who work in all these different fields about their work and clarifying complicated topics for everybody, I think that’s fantastic work. Thank you. And next time, the interview we do will be more technical. We can even use a whiteboard. We’ll see. We’ll see. Anyway, it’s been a joy. Thank you so much for the invitation. Thank you. And also, I want to thank our partner, The Economist. I also discovered last year that external links are hugely important in the algorithm, meaning that the more you share on Twitter, for example on Facebook or even on Reddit, et cetera, it shows YouTube, hey, people are talking about this content outside of YouTube which in turn helps massively in distribution on YouTube. Secondly, there’s a remarkably active discord and subreddit for Theories of Everything where people explain their toes. They respectfully disagree on theories and build as a community our TOE links for both in the description. Fourthly, you should know that this podcast is on iTunes. It’s on Spotify. It’s on all podcast platforms. All you have to do is type Theories of Everything and you’ll find it. I personally benefit from re-watching lectures and podcasts. I also read in the comments that TOE listeners also benefit from replaying. So, consider re-listening instead on those platforms like iTunes and Spotify and Google Podcasts, or whatever podcast software you use. And finally, if you would like to support more conversations like this, and more content like this, consider visiting patreon.com and slashing CurtJaimungal and donating whatever you like. There’s also PayPal. There’s also crypto. There’s also just joining on YouTube. Again, bear in mind that the support from patrons and from you is what allows me to work on TOE full time. You also get early access to add free episodes, whether audio or video. It’s audio in the case of Patreon, and video in the case of YouTube. For instance, this episode you are listening to right now was released a few days ago. Every dollar helps far more than you think. And in either case, your viewership is kind enough. Thank you very much.