2nd Law of Thermodynamics explained: Things get more random over time | Stephen Wolfram

You wrote the blog post, "The 50-Year Quest: My Personal Journey," good title. My personal journey with the second law of thermodynamics. So what is this law, and what have you understood about it in the 50-year journey you had with it? Right. So, the second law of thermodynamics, sometimes called the law of entropy increase, is this principle of physics that says—well, my version of it would be—things tend to get more random over time. A version of it—that there are many different sorts of formulations of it—that are things like heat doesn't spontaneously go from a hotter body to a colder one. When you have uh, mechanical work, kind of gets dissipated into heat. You have friction, and and uh, kind of when you systematically move things, eventually there'll be—they'll be sort of—that the energy of moving things gets kind of ground down into heat.

So people first sort of paid attention to this back in the 1820s, when steam engines were a big thing, and the big question was how efficient could a steam engine be. And there's this chap called Sadi Carnot, who was a French engineer; actually, his father was a sort of elaborate mathematical engineer in France. Um, but he figured out these—this kind of rules for how uh, kind of the the efficiency of—of the possible efficiency of something like a steam engine. And in sort of a side part of what he did was this idea that mechanical energy tends to get dissipated as heat; that you—that you end up going from sort of systematic mechanical motion to this kind of random thing. Well, at that time, nobody knew what heat was. At that time, people thought that heat was a fluid—like they called it caloric—and it was a fluid that kind of—kind of was absorbed into substances, and when—when heat—when one hot thing would transfer heat to a colder thing, that this fluid would flow from the hot thing to the colder thing. But anyway, then by the—by the 1860s, people had uh, kind of come up with this idea that systematic energy tends to degrade into kind of random heat, that would uh—that—that could then not be easily turned back into systematic mechanical energy. Um, and then that—that quickly became sort of a global principle about how things work.

Question is, why does it happen that way? So, you know, let's say you have a bunch of molecules in a box, and they're arranged—these molecules arranged in a very nice sort of uh, flotilla of molecules in one corner of the box, and then what you typically observe is that after a while these molecules were kind of randomly arranged in the box. Question is, why does that happen? And people for a long, long time tried to figure out—is there, from the laws of mechanics that just determine how these molecules—that say these molecules like hard spheres bouncing off each other—from the laws of mechanics that describe those molecules, can we explain why it tends to be the case that we see things that are orderly sort of degrade into disorder? We tend to see things that uh, you know, you—you uh, you scramble an egg; you—um—that you know, you take something's quite ordered, and you—you disorder it, so to speak. That's the thing that sort of happens quite regularly. Or you—you put some ink into water, and it will eventually spread out and—and fill up—you know, fill up the water. Um, but you don't see those little particles of ink in the water all spontaneously kind of arrange themselves into a big blob, and then you know, jump out of the water or something. Um, and uh, so the question is, why do things happen in this kind of irreversible way, where you go from order to disorder? Why does it happen that way?

And so throughout, in the later part of the 1800s, a lot of work was done on trying to figure out—can one derive this principle, this second law of thermodynamics, this law about the the dynamics of heat, so to speak—can one derive this from uh, from some fundamental principles of mechanics? You know, in the laws of thermodynamics, the first law is basically the law of energy—energy conservation—that the total energy associated with heat plus the total energy associated with the mechanical kinds of things plus other kinds of energy—that that total is constant—and that became a pretty well-understood principle. But the second law of thermodynamics was always mysterious—like, why does it work this way? Can it be derived from underlying mechanical laws?

And so when I was uh—uh, well, 12 years old actually, I had gotten interested—well, I've been interested in in space and things like that, because I thought that was kind of the—the future and um, interesting sort of technology and so on. And for a while, kind of uh, you know, every deep space probe was sort of a personal friend type thing, and I knew all—all kinds of characteristics of it and uh—uh, was kind of writing up all these—all these things when I was—no, I don't know, eight, nine, ten years old and so on. And then I—I got interested—from being interested in kind of spacecraft, I got interested in, like, how do they work, what all the instruments on them and so on, and that got me interested in physics, which was just as well, because if I'd stayed interested in space in the—you know, mid-to-late 1960s, I would have had a long wait before you know, space really blossomed as a—as an area. But anyway, right, I got interested in physics. And uh, then—well, the actual sort of detailed story is when I—when I kind of graduated from elementary school at age 12—that's the time when in England where you finish elementary school—I sort of—my—my gift, sort of—I suppose more or less for myself—was I got um, this collection of um, physics books, which was some college physics course—of college physics books—and volume five about statistical physics. It has this picture on the cover that shows a bunch of kind of idealized molecules sitting in one side of a box, and then it has a series of frames showing how these molecules sort of spread out in the box. And I thought, that's pretty interesting—you know, what causes that? And you know, I read the book, and—and the book—the book, actually, one of the things that was really significant to me about that was the book kind of claimed—although I didn't really understand what it said in detail—it kind of claimed that this sort of principle of physics was derivable somehow. And you know, other things I'd learned about physics, it was all like—it's a fact that energy is conserved; it's a fact that relativity works or something—not—it's something you can derive from some fundamental sort of—it has to be that way as a matter of kind of—of mathematics or logic or something. So it was sort of interesting to me that there was a thing about physics that was kind of inevitably true and derivable, so to speak. And so I think that—so then I was like—this picture on this book, and I was trying to understand it. And so that was actually the first serious program that I wrote for a computer—was probably in 1973—um, written for this computer the size of a desk—program with paper tape and so on—and I tried to reproduce this picture on the book, and I didn't succeed.

What was the failure mode there? Like, what do you mean you didn't succeed? So it's a bunch of—looked like it didn't look like—okay. So what happened is—okay, many years later I learned how the picture on the book was actually made, and that it was actually kind of a fake, but I didn't know that at that time. Um, but uh—and that picture was actually a very high-tech thing when it was made in the beginning of the 1960s—was made on the largest supercomputer that existed at the time—and uh, even so it couldn't quite simulate the thing that it was supposed to be simulating. But anyway, I didn't know that until many, many years later. So at the time it was like—you have these balls bouncing around in this box, but I was using this computer with eight kilowatts of memory—there were 18-bit words of memory words—okay—so it was um, whatever, 24 kilobytes of memory—um, and it had—you know, it had these instructions—I probably still remember all of its machine instructions—um, and it didn't really like dealing with floating-point numbers or anything like that. And so I had to simplify this—this model of—of you know, particles bouncing around a box. And so I thought, well, I'll put them on a grid, and I'll make—you know, make the things just sort of move one square at a time and so on. And so I did the simulation, and the result was it didn't look anything like the actual pictures on the book. Now many years later—in fact, very recently—I realized that the thing I'd simulated was actually an example of a whole sort of computational irreducibility story that I absolutely did not recognize at the time. At the time it just looked like it did something random and it looks wrong, as opposed to it did something random and it's super interesting that it's random. Um, but I didn't recognize that at the time. And so as it was at the time, I kind of—I got interested in particle physics, and I got interested in—in other kinds of physics, and but this whole second law of thermodynamics thing—this idea that sort of orderly things tend to degrade into disorder—continued to be something I was really interested in, and I was really curious for the whole universe—why doesn't that happen all the time? Like, we start off at the—in the Big Bang—at the beginning of the universe—was this thing that seems like it's this very disordered collection of—of stuff, and then it spontaneously forms itself into galaxies and creates all of complexity and order in the universe. And so I was very curious how that happens, and I—but I was always kind of thinking—this is kind of somehow the second law of thermodynamics is behind it—trying to sort of pull things back into disorder, so to speak—and how was order being created? And so actually I was—was interested—this is probably now 1980—I got interested in kind of this—you know, galaxy formation and so on in the universe. I also at that time was interested in neural networks, and I was interested in kind of how—how brains make complicated things happen and so on.

Okay, what's the connection between the formation of galaxies and how brains make complicated things happen? Because they're both a matter of how complicated things come to happen from simple origins—yeah—from some sort of known origins. I had the sense that that what I was interested in was kind of in all these different—this sort of different cases of where complicated things were arising from rules. And you know, I also looked at snowflakes and things like that. Um, I was curious—and fluid dynamics in general—I was just sort of curious about how does complexity arise? And the—the thing that I didn't—you know, it took me a while to kind of realize that there might be a general phenomenon. You know, I sort of assumed—oh, there's galaxies over here, there's brains over here—that they're very different kinds of things. And so what happened—this is probably 1981 or so—I decided—okay, I'm—I'm going to try and make the minimal model of how these things work. Yes, it was sort of an interesting experience because I had built—starting in 1979—I built my first big computer system—to think—called SMP—symbolic manipulation program—it's kind of a runner of modern morpheme language with many of the same ideas about symbolic computation and so on. Um, but the thing that was very important to me about that was—you know, in building that language, I'd basically tried to figure out what were the sort of—what were the relevant computational primitives, which have turned out to stay with me for the last 40-something years. But it was also important because in building a language was a very different activity from natural science, which is what I've mostly done before; because in natural science you start from the phenomena of the world, and you try and figure out—so how can I make sense of the phenomena of the world? And you know, kind of the world presents you with what it has to offer, so to speak, and you have to make sense of it. When you build a—a, you know, computer language or something, you are creating your own primitives, and then you say—come—so what can you make from these? Sort of the opposite way around from what you do in natural science. But I'd had the experience of doing that, and so I was kind of like—okay, what happens if you sort of make an artificial physics? What happens if you just make up the rules by which systems operate? And then I was thinking—you know, for all these different systems, whether it was galaxies or brains or whatever—what's the absolutely minimal model that kind of captures the things that are important about those systems? The computational primitives of that system? Yes. And so that's what ended up with the cellular automata, where you just have a line of black and white cells; you just have a rule that says—you know, given a cell and its neighbors, what will the color of the cell be on the next step? And you just run it in a series of steps. And the sort of the ironic thing is that cellular automata are great models for many kinds of things, but galaxies and brains are two examples where they do very, very badly; they're really irrelevant to those two.

Is there a connection to the second law of thermodynamics and cellular automata? Oh, yes. So the things you—the things you've discovered about cellular automata? Yes. Okay. So when I first started studying cellular automata, my first papers about them were—you know, the first sentence was always about the second law of thermodynamics; it was always about—how does order manage to be produced even though there's a second law of thermodynamics which tries to pull things back into disorder? And I kind of—my early understanding of that had to do with—these are intrinsically irreversible processes in cellular automata—that—that form—it's kind of—conform orderly structures even from random initial conditions. But then what I realized—this was—well, actually, it's—it's one of these things where it was a discovery that I should have made earlier but didn't. So you know, I had—I've been studying cellular automata—what I did was the sort of most obvious computer experiment—you just try all the different rules and see what they do. It's kind of like—you know, you've invented a computational telescope—you just pointed at the most obvious thing in the sky, and then you just see what's there. And so I did that, and I—you know, I was making all these pictures of—of how cellular automata work, and—and I studied these pictures—I studied them in great detail. There was—you can number the rules for cellular automata, and one of them is—you know, rule 30. So I made a picture of rule 30 back in 1981 or so, and rule 30—well, it's—and at the time I was just like—okay, it's another one of these rules—I don't really—it happens to be asymmetric—left-right asymmetric—and it's like—let me just consider the case of the symmetric ones just to keep things simpler, etc., etc., etc.—and I just kind of ignored it. Yeah. And then sort of in—and actually in 1984, strangely enough, I ended up having an early laser printer which made very high-resolution pictures, and I thought—I'm going to print out an interesting—you know, I want to make an interesting picture—let me take this rule 30 thing and just make a high-resolution picture of it. I did, and it has this very remarkable property that it's rule is very simple—you started off just from one black cell at the top, and it makes this kind of triangular pattern. But if you look inside this pattern, it looks really random; there's—you know, you look at the center column of cells, and—you know, I studied that in great detail, and it's—so far as one can tell—it's completely random. And it's kind of a little bit like digits of pi—once you—you know—you know the rule for generating the digits of pi, but once you've generated them—you know, 3.14159, etc.—they seem completely random. And in fact, I put up this prize back in—what was it?—2019 or something—for—prove anything about the sequence. Basically, has anyone been able to do anything on that? Uh, people have sent me some things, but it's—you know, I don't know how these problems are—I mean, I was kind of spoiled because I—in 2007—I put up a prize for uh, determining whether a particular Turing machine—that I thought was the simplest candidate for being a universal Turing machine—determine whether it is or isn't a universal Turing machine—and somebody did a really good job of—of winning that prize and proving that it was a universal Turing machine in about six months. And so I—you know, I didn't know whether that would be one of these problems that was out there for hundreds of years, or whether—in this particular case—a young chap called Alex Smith—um—you know, nailed it in six months. And so with this rule 30 collection, I don't really know whether these are things that are 100 years away from being able to—to get, or whether somebody's going to come and do something very clever. It's such a—means—like—for my last theorem—essentially—rule 30 is such a simple formulation—it feels like anyone can look at it and understand it. Yeah. And feel like it's within grasp to be able to predict something—to—to direct some kind of law, right? It allows you to predict something about this middle column of rule 30, right? But you know—this is—yeah—you can't—yeah—right—this is the intuitional surprise of computational irreducibility and so on—that even though the rules are simple, you can't tell what's going to happen, and you can't prove things about it. And I think—so—so anyway, the—the—the thing—I—I still—started in 1984 or so—I started realizing there's this phenomenon that you can have very simple rules—they produce apparently random behavior. Okay, so that's a little bit like the second law of thermodynamics, because it's like—you have this simple initial condition—you can—you know, readily see that it's very—you know, you can describe it very easily—and yet it makes this thing that seems to be random. Now, it turns out—thus some technical detail about the secular thermodynamics and about the idea of reversibility—when you have a—if you have kind of a—a—a movie of two—you know, billiard balls colliding, and you see them collide and they bounce off, and you run that movie in reverse, you can't tell which way was the forward direction of time and which way was the backward direction of time when you're just looking at individual billiard balls. By the time you've got a whole collection of them—you know, a million of them or something—then it turns out to be the case—and this is the—the sort of the—the mystery of the second law—that the orderly thing—you start with the orderly thing, and it becomes disordered, and that's the forward direction in time. And the other way around—of it starts disordered and becomes ordered—you just don't see that in the world. Now, in principle, if you—you know—if you sort of traced the detailed motions of all those molecules backwards, you would be able to—it—it will—it will—the reverse of time makes—you know—as you—as you go forwards in time, order goes to disorder; as you go backwards in time, order goes to disorder perfectly. So, yes, right. So the—the mystery is—why is it the case that—one version of the mystery is—why is it the case that you never see something which happens to be just the kind of disorder that you would need to somehow evolve to order? Why does that not happen? Why do you always just see order goes to disorder, not the other way around? So the thing that I—I kind of realized—I started realizing in the 1980s—is kind of like—it's a bit like cryptography; it's kind of like—you start off from this—this key that's pretty simple, and then you kind of run it, and you can get this—you know—complicated random mess. And uh, the thing that—that—well, I sort of started realizing back then was that the second law is kind of a—a story of computational irreducibility; it's a story of—you know, what seems—you know, what—what we can describe easily at the beginning—we can only describe with a lot of computational effort.

At the end, okay. So now we come many, many years later, and um, uh, I was trying to sort of uh, well, having done this big project to understand fundamental physics, I realized that sort of a key aspect of that is understanding what observers are like. And then I realized that the second law of thermodynamics is the same story as a bunch of these other cases. Um, it is a story of a computationally bounded Observer trying to observe a computationally irreducible system. So it's a story of, you know, underneath the molecules are bouncing around; they're bouncing around in this completely uh, determined way, determined by rules. But the point is that that we, as computationally bounded observers, can't tell that there were these sort of simple underlying rules. To us, it just looks random.

And when it comes to this question about, can you prepare the initial state so that um, you know, the disordered thing is, you know, how exactly the right disorder to make something orderly, a computationally bounded Observer cannot do that. We'd have to have done all of this sort of irreducible computation to work out very precisely what this disordered State—what the exact right disordered state is—so that we would get this ordered thing produced from it. What does it mean to be a computationally bounded Observer observing a computational reducible system? So the computationally bounded—is there something formal you can say there, right? So it means, okay, you can you can talk about Turing machines, you can talk about computational uh, complexity Theory, and uh, you know, uh, polynomial time computation and things like this. There are a variety of ways to make something more precise, but I think it's more useful—the intuitive version of it is more useful.

Yeah, which is basically just to say that, you know, how much computation are you going to do to try and work out what's going on? And the answer is you're not allowed to do a lot of—we're not able to do a lot of computation when we, you know, we've got, you know, in this room there will be a trillion trillion trillion molecules—a little bit less—it's a big room, right? And uh, you know, at every moment, you know, that every microsecond or something, these molecules—molecules are colliding, and that's a lot of computation that's getting done. And the question is, in our brains we do a lot less computation every second than the computation done by all those molecules. If there is computational irreducibility, we can't work out in detail what all those molecules are going to do. What we can do is only a much smaller amount of computation. And so the second law of Thermodynamics is this kind of interplay between the underlying computational irreducibility and the fact that we, as preparers of initial States or as measures of what happens, are, you know, are not capable of doing that much computation.

So to us, another big formulation of the second law of thermodynamics is this idea of the law of entropy increase—the characteristic that this universe—the entropy—seems to be always increasing. What does that show to you about the evolution of—yes, okay. And that's very confused in the history of thermodynamics because entropy was first introduced by a guy called Rudolf Clausius, and he did it in terms of heat and temperature. Okay. Subsequently, it was reformulated by a guy called Ludwig Boltzmann, um, and uh, he formulated it in a much more kind of combinatorial type way, but he always claimed that it was equivalent to Clausius's thing. And then in one particular simple example it is, but that connection between these two formulations of entropy—they've never been connected. I mean, there's really—so okay, so the more general definition of entropy due to Boltzmann is is the following thing. So you say I have a system and has many possible configurations—molecules can be in many different Arrangements, etc., etc., etc. If we know something about the system—for example, we know it's in a box, it has a certain pressure, it has a certain temperature—we know these overall facts about it, then we say how many microscopic configurations of the system are possible given those overall constraints, um, and the entropy is the logarithm of that number. That's the definition, and that's the kind of the general definition of entropy that turns out to be useful.

Now, in Boltzmann's time, he thought these molecules can be placed anywhere you want. He didn't think—a—he said, oh, actually we can make it a lot simpler by having the molecules be discrete. Well, actually, he didn't know molecules existed, right? And in those—in his time, 1860s and so on—the idea that matter might be made of discrete stuff had been floated ever since ancient Greek times, but it had been a long time debate about, you know, is matter discrete, is it continuous? At the moment, at that time, people mostly thought that matter was continuous, and um, it was all confused with this question about what heat is, and people thought heat was this fluid, and um, it was it was a big big model, and um, the uh, and this—but Boltzmann said, let's assume there are discrete molecules, let's even assume they have discrete energy levels, let's say everything is discrete, then we can do sort of combinatorial mathematics and work out how many configurations of these things they would be in the box, and we can say we can compute the entropy quantity. But he said—said, but of course it's just a fiction that these things are discrete. So he said this is an interesting piece of History, by the way, that that, you know, that was at that time people didn't know molecules existed.

There were other hints from from looking at uh, kind of chemistry that there might be discrete atoms and so on, just from the combinatorics of, you know, two hydrogens and one oxygen make water—you know, two two amounts of hydrogen plus one amount of oxygen together make water—things like this. But it wasn't known that discrete molecules existed. And in fact, the um, uh, people, you know, it wasn't until the beginning of the 20th century that Brownian motion was the final giveaway. Brownian motion is, you know, you look under a microscope at these little pieces from pollen grains; you see they're being discreetly kicked, and those kicks are water molecules hitting them, and they're discrete. Um, and uh, in fact, it was um, it was really quite interesting history. I mean, Boltzmann had worked out how things could be discrete and have basically invented something like quantum theory in in the 1860s, and uh, but he just thought it wasn't really the way it worked. And then just a piece of physics history because I think it's kind of interesting. In 1900, this guy called Max Planck, who'd been a long time thermodynamics person who was trying to—everybody was trying to prove the second law of thermodynamics, including Max Planck, and Max Planck believed that radiation—like electromagnetic radiation—somehow the interaction of that with matter was going to prove the second law of thermodynamics. But he had these experiments that people had done on black body radiation, and there were these curves, and you couldn't fit the curve based on his idea for how radiation interacted with matter. Those curves—you couldn't figure out how to fit those curves—except he noticed that if he just did what Boltzmann had done and assumed that electromagnetic radiation was discrete, he could fit the curves. He said, but you know, this is just a, you know, it just happens to work this way. Then Einstein came along and said, well, by the way, you know, uh, the electromagnetic field might actually be discrete; it might be made of photons, and then that explains how this all works. And that was, you know, in 1905—that was that was how um, kind of that was how quantum—that piece of quantum mechanics got started.

Kind of interesting—interesting piece of History. I didn't know until I was researching this recently. In 1904 and 1903, Einstein wrote three different papers, and uh, so you know, just sort of a well-known physics history. In 1905, Einstein wrote these three papers: one introduced relativity Theory, one explained Brownian motion, and one introduced basically photons. So kind of, you know, kind of a a big deal year for physics and for Einstein. But in the years before that, he'd written several papers, and what were they about? They were about the second law of thermodynamics, and they were an attempt to prove the second law of thermodynamics, and they're nonsense. And so I I had no idea that he'd done this. Interesting—oh, neither. And in fact, what he did—those three papers in 1905—well, not so much the relativity paper, the one on Brownian motion, the one on photons—both of these were about the story of sort of making the world discrete. Um, and then he got those like that idea from Boltzmann. Yeah. Um, but Boltzmann didn't think, you know, Boltzmann kind of died believing, you know, he said—but he has a quote actually, you know, uh, you know, in the end things are going to turn out to be discrete, and I'm going to write down what I have to say about this because uh, uh, you know, eventually the stuff will be rediscovered, and I want to leave, you know, what I can about how things are going to be discrete. But you know, um, I think he has some quotes about how, you know, one person can't stand against the tide of history in um, uh, in saying that, you know, matter is discrete. So so he stood by his guns—it doesn't matter—is discrete. Yes, he did. And and the, you know, what's interesting about this is uh, at the time everybody, including Einstein, kind of assumed that space was probably going to end up being discrete too, but that didn't work out technically because it wasn't consistent with relativity Theory; it didn't seem to be. And so then in the history of physics, even though people had determined that matter was discrete, electromagnetic field was discrete, space was a holdout of not being discrete. And in fact, Einstein, 1916, has this nice letter he wrote where he says, in the end it will turn out space is discrete, but we don't have the mathematical tools necessary to figure out how that works yet. And so, you know, I think it's kind of cool that 100 years later we do. Yes. For you, you're pretty—pretty sure that at every layer of reality it's discrete, right? And that space is discrete, and that the—I mean, and in fact, one of the things I realized recently is this kind of theory of heat that um, uh, that the um, you know, that heat is really this continuous fluid um, it's it's kind of like uh, the the, you know, the caloric theory of heat, which turns out to be completely wrong because actually heat is the motion of discrete molecules. Unless you know there are discrete molecules, it's hard to understand what heat could possibly be. Well, you know, I think space is is discrete, and the question is kind of what's the analog of the mistake that was made with caloric in the case of space. And so I'm—my my current guess is that dark matter is—as I've—my little sort of aphorism of the of the last few months has been, you know, dark matter is the caloric of our time—that is, it will turn out the dark matter is a feature of space, and it is not a bunch of particles. You know, at the time when when people were talking about heat, they knew about fluids, and they said, well, heat must just be another kind of fluid because that's what they knew about. Yes. But now people know about particles, and so they say, well, what's dark matter? It's not—it's not—it just must be particles. So what could dark matter be as a feature of space? Oh, I don't know yet. All right. Um, I mean, I think the the thing I'm really—one of the things I'm hoping to be able to do is to find the analog of Brownian motion in space. So in other words, Brownian motion was was seeing down to the level of an effect from individual molecules. And so in the case of space, you know, most of the things—the things we see about space so far—just everything seems continuous. Brownian motion had been discovered in the 1830s, and it was only identified—what it was—what it was—the results of—by Smoluchowski and Einstein at the beginning of the 20th century. And you know, Dark Matter was—was discovered—that phenomenon was discovered 100 years ago—you know, the rotation curves of galaxies don't follow the luminous matter—that was discovered 100 years ago. And I think, you know, that I I wouldn't be surprised if there isn't an effect that we already know about that is kind of the analog of Brownian motion that reveals the discreetness of space. And in fact, we're beginning to have some guesses—we have some some evidence that black hole mergers work differently when there's discrete space, and there may be things that you can see in gravitational wave signatures and things associated with the discreetness of space. But this is kind of—for me it's kind of—it's kind of interesting to see this sort of recapitulation of the history of physics where people, you know, vehemently say, you know, matter is continuous, electromagnetic field is continuous, and turns out it isn't true. And then they say space is continuous. But—but so, you know, entropy is the number of states of the system consistent with some constraint. Yes. And the the thing is that if you have—if you know in great detail the position of every molecule in the gas, the entropy is is always zero because there's only one possible State—the configuration of molecules in the gas—the molecules bounce around, they have a certain rule for bouncing around—there's just one state of the gas evolves to one state of the gas and so on. But it's only if you don't know in detail where all the molecules are that you can say, well, the entropy increases because the things we do know about the molecules—there are more possible microscopic states of the system consistent with what we do know about where the molecules are. And so the question of whether—so people—this sort of paradox in a sense of, oh, if we knew where all the molecules were, the entropy wouldn't increase. There was this idea introduced by by Gibbs in the early 20th century—well, actually the very beginning of the 20th century—as a physics Professor—an American physics Professor—was sort of the first distinguished American physics Professor um, at Yale, um, and he he introduced—reduced this idea of coarse graining—this idea that, well, you know, these molecules have a detail where they're bouncing around, but we can only observe a coarse-grained version of that. But the confusion has been—nobody knew what a valid coarse graining would be. So nobody knew that whether you could have the coarse graining that very carefully was sculpted in just such a way that it would notice that the particular configurations that you could get from the simple initial condition, you know, they fit into this coarse graining, and the coarse graining very carefully observes that—why can't you do that kind of very detailed precise coarse graining? The answer is because if you are a computationally bounded Observer and the underlying Dynamics is computationally irreducible—that's—that's what defines possible coarse grainings—is what a computationally bounded Observer can do. And it's the—it's the fact that a computationally bounded Observer uh, is is forced to look only at this kind of coarse-grained version of what the system is doing—that's why—and and because the—what's—what's going on underneath is—it's kind of filling out this—this—the—the different possible—you're ending up with something where the sort of underlying computational irreducibility is—your—if—if all you can see is what the coarse-grained result is—with—with a sort of computationally bounded observation—then inevitably there are many possible underlying configurations that are consistent with that.

Just to clarify, basically any Observer that exists inside the universe is going to be computationally bounded. No—any Observer like us—I don't know—I can't say like us. What do you mean—what do you mean like us? Well, humans with finite Minds. You're including the tools of science. Yeah, yeah. I mean, and and as we, you know, we have more precise—and by the way, there are little sort of microscopic violations of the second law of thermodynamics that you can start to have when you have more precise measurements of where precisely molecules are. But for uh, for a large scale—when you have enough molecules—we don't have—you know, we're not tracing all those molecules, and we just don't have the computational resources to do that. And it wouldn't be uh, you know, I think the the to imagine what an observer who is not computationally bounded would be like—it's an interesting thing because okay, so what does computational boundedness mean? Among other things, it means we conclude that definite things happen. We go—we take all this complexity of the world and we make a decision—we're going to turn left or turn right—and that is kind of reducing all this kind of uh, detail into—we're observing it—we're—we're sort of crushing it down to this this one thing. Yeah. And and that if we didn't do that uh, we wouldn't—we wouldn't have all this sort of symbolic structure that we build up that lets us think things through with our finite Minds. We'd be instead, you know, we'd be just—we'd be sort of one with the universe. Yeah. So content to not simplify. Yes. If we didn't simplify, then we wouldn't be like us; we would be like the universe—like the the intrinsic universe—but not having experiences like the experiences we have where we, for example, conclude that definite things happen—we, you know, we we sort of have this this uh, uh notion of being able to make—make sort of narrative statements. Yeah. I wonder if it's just like you imagined as a thought experiment what it's like to be a computer. I wonder if it's possible to try to begin to imagine what it's like to be an unbounded computational—well, okay, so here's—here's how that I think plays out. Vibrations suck. Yeah. So I mean, in this—we talk about this rouliad—the space of all possible computation. Yes. And this idea of, you know, being at a certain place in the rouliad, which corresponds to sort of a certain way of—of rep—of a certain set of computations that you're representing things in terms of. Okay. So as you expand out in the rouliad—as you kind of encompass more possible views of the universe—as you encompass more possible kinds of computations that you can do—eventually you might say that's a real win—you know, we're colonizing the rouliad—we're—we're—we're building out more paradigms about how to think about things—and eventually you might say we—we won all the way—we managed to colonize the whole rouliad. Okay. Here's the problem with that. The problem is that the notion of existence—coherent existence—requires some kind of specialization. By the time you are the whole rouliad—by the time you cover the whole rouliad—in no useful sense do you coherently exist. So in other words—in inches—the notion of existence—the notion of what we think of as as definite existence—requires this kind of specialization—requires this kind of idea that we are—we are not all possible things—we are a particular set of things—and that's kind of how we uh—that that's kind of what—what makes us have a coherent existence. If we were spread throughout the rouliad, we would not—there would be no coherence to the way that we work—we would work in all possible ways, and that wouldn't be kind of a notion of identity. We wouldn't have this notion of kind of uh, uh, of—of—of coherent identity. I am geographically located somewhere—exactly—precisely in the rouliad—therefore I am. Physical space—you're in a certain place in real space, and if—if you uh, if you are sufficiently spread out, you are no longer coherent, and you no longer have—I mean, in—in the—in our perception of what it means to exist and to have experience—it doesn't happen. Now—so therefore—so to exist means to be computationally bounded. I think so—to exist in the way that we think of ourselves as existing—yes—the very active existence is like operating in this place that's computationally reducible. So this is just a giant mess of things going on that you can't possibly predict, but nevertheless, because of your limitations, you have an imperative of—like—what is it—an imperative or a skill set to simplify—or an ignorance—a sufficient love. Okay. So the thing which is not obvious is that you are taking a slice of all this complexity—just like we have all of these molecules bouncing around in the room, but all we notice is, you know, the—the—the kind of the flow of the air or the pressure of the air—we're just noticing these particular things. And the the big interesting thing is that there are rules—there are laws that govern those big things that we—we observe. Yeah. So it's not obvious—it's amazing because it doesn't.

Feel like it's a slice? Yeah, well, right, not a slice. Well, it's like uh, it's like an abstraction. Yes, but I mean, the fact that the gas laws work, that we can describe pressure, volume, etc., etc., etc., we don't have to go down to the level of talking about individual molecules—that is a non-trivial fact.

And, and here's the thing that I sort of find exciting, as far as I'm concerned: the fact that there are certain aspects of the universe—so you know, we think space is made ultimately of these atoms of space and these typographs and so on, and we think that uh, but we nevertheless perceive the universe at a large scale to be like continuous space and so on. Um, we uh, in quantum mechanics, we think that there are these many threads of time, these many threads of history, yet we kind of span—so, so you know, in in quantum mechanics, in our models of physics, there are these—time is not a single thread; time breaks into many threads. They branch; they merge. And but we we are part of that branching, merging universe, right? And so our brains are also branching and merging. And so when we perceive the universe, we are branching brains perceiving a branching universe. Yeah.

And so the fact that the claim that we exp—we believe that we are persistent in time—we have this single thread of experience—that's the statement that somehow we managed to aggregate together those separate threads of time that are separated in in the operation of in the fundamental operation of the universe. So just as in space we're averaging over some big region of space and we're looking at many, many of the aggregate effects of many atoms of space, so similarly in what we call branchial space, the space of these these quantum branches, we are effectively averaging over many different branches of possible of histories of the universe. And so in in thermodynamics we're averaging over many configurations of, you know, many, many possible positions of molecules. Yeah.

So what what we see here is—so the question is, when you do that averaging for space, what are the aggregate laws of space? When you do that averaging of a branchial space, what are the aggregate laws of branchial space? When you do that averaging over the molecules and so on, what are the aggregate laws you get? And this is this is the thing that I I think is just amazingly, amazingly neat—that there are aggregate laws at all. Well, yes, but the question is, what are those aggregate laws? So the answer is: for space, the aggregate laws are Einstein's equations for gravity, for the structure of space-time; for branchial space, the aggregate laws are the laws of quantum mechanics; and for uh the case of of molecules and things, the aggregate laws are basically the second law of thermodynamics. And so the um—though that's the—and the things that follow from the second law of thermodynamics. And so what that means is that the three great theories of 20th-century physics, which are basically general relativity, the theory of gravity, uh, quantum mechanics, and statistical mechanics, which is what kind of grows out of the second law of thermodynamics—all three of the great theories of 20th-century physics are the result of this interplay between computational irreducibility and the computational boundedness of observers.

And you know, for me this is really neat because it means that all three of these laws are derivable. So we used to think that, for example, Einstein's equations were just sort of a wheel-in feature of our universe—that they could be—the universe might be that way; it might not be that way. Quantum mechanics is just like, well, it just happens to be that way. And the second law, people kind of thought, well, maybe it is derivable. Okay, what turns out to be the case is that all three of the fundamental principles of physics are derivable, but they're not derivable just from mathematics, they require—or just from some kind of logical computation—they require one more thing: they require that the observer, that the thing that is sampling the way the universe works, is an observer who has these characteristics of computational boundedness, of belief, and persistence in time. And so that that means that it is the nature of the observer—you know, the rough nature of the observer, not the details of, oh, we got two eyes and we've observed photons of this frequency and so on—but the the the the kind of the very coarse features of the observer—um, then imply these very precise facts about physics. And it's it's I think it's amazing.

So if we just look at the actual experience of the observer—that we experience this reality—it seems real to us. And you're saying because of our bounded nature it's actually all an illusion? It's a simplification. Yeah, it's a simplification. Right. What's—what's—so you don't think a simplification is an illusion? No, I mean it's it's—well, I don't know. I mean, what underneath—okay, that's an interesting question. Um, what's real? And that relates to the whole question of why does the universe exist? And um, you know, what is the difference between reality and a mere representation of what's going on? Yes, we experience the representation. Yes, but the the question of—so so one question is, uh, you know, why is there a thing which we can experience that way? And the answer is because this rouliad object, which is this entangled limit of all possible computations—there is no choice about it—it has to exist; it has to—there has to be such a thing. It is in in the same sense that you know, two plus two—if you define what two is and you define pluses and so on—two plus two has to equal four. Similarly, this rouliad, this limit of all possible computations, just has to be a thing. You—that is, once you have the idea of computation, you inevitably have the rouliad. Yeah. You're gonna have to have a rouliad. Yeah, yeah, right. And and what's important about it—there's just one of it. It's it's it's just this unique object, and that unique object necessarily exists. And then the question is what—and then we—uh—once once you know that we are sort of embedded in that and taking samples of it—that it's sort of inevitable that there is this thing that we can perceive that is, you know, the the our perception of kind of physical reality necessarily is that way, given that we are observers with the characteristics we have.

So in other words, the fact that the fact that the universe exists is—it's actually—it's almost like it's, you know, to think about it almost theologically, so to speak. And I I really—it's it's funny because a lot of the questions about the existence of the universe and so on, they they transcend what kind of the science of the last few hundred years has really been concerned with. The science of the last few hundred years hasn't thought it could talk about questions like that. Yeah. Um, and uh, but I think it's kind of—and so a lot of the kind of arguments of, you know, does God exist, you know, is it obvious that—I think it in some sense, in some representation, it's sort of more more obvious that uh that something sort of bigger than us exists than that we exist. And we are, you know, our existence and as observers the way we are is sort of a contingent thing about the universe. And it's more inevitable that the whole the whole universe, kind of the whole set of all possibilities, exists. But but this question about, you know, is is it real or is it an illusion—you know, all we know is our experience. And so the fact that—well, our experience is this absolutely microscopic piece of sample of the rouliad, and we're—um—and and you know, there's this this point about, you know, we might sample more and more of the rouliad, we might learn more and more about—we might learn, you know, like like different areas of physics, like quantum mechanics, for example—the fact that it it was discovered I think is closely related to the fact that electronic amplifiers were invented that allowed you to take a small effect and amplify it up, which hadn't been possible before. You know, microscopes have been invented that magnify things and so on, but they're—you know, having a very small effect and being able to magnify it was sort of a new thing that allowed one to see a different sort of aspect of the universe and let one discover this kind of thing. So you know, we can expect that in the rouliad there's an infinite collection of new things we can discover; there's—there's in fact computational irreducibility kind of guarantees that there will be an infinite collection of kind of, you know, pockets of reducibility that can be discovered. Boy, would it be fun to take a walk down the rouliad and see what kind of stuff we find there.