Transcription
And the big lesson of machine learning that we particularly learned in 2011 was, in a neural net, if you bash it hard enough, it will learn stuff. And that wasn't obvious. Nobody knew that you could get a deep learning, a deep neural net to recognize images of cats and dogs and things. And sort of by accident, it was discovered that if you just leave the thing learning long enough, if you leave it training long enough, it comes out and it's actually succeeded in learning something.
So that's a new piece of intuition that you can have a system like a neural net. A neural net is much more complicated than one of these cellular automata. You can have a system like that. You just keep bashing it, a quadrillion times or something, and eventually it will successfully achieve some fitness function, will learn something, will do the thing you want it to do. So that was a piece of new intuition.
So I thought, let me just run these. I was writing something about foundations of machine learning, and I thought, let me just try this experiment. I thought I'd tried it in the 1980s, and it hadn't worked, but now we know something different from our studies of neural nets. Let me try running it a bit longer. And by golly, it worked. And I felt a bit silly for not having discovered this any time in the intervening 40 years or something, but nevertheless, it was really cool that it worked.
And that meant that one could actually see, for the first time, in a more detailed way, how natural selection operates. And what's really going on in the end is computational irreducibility lets one go from these quite simple rules to these very elaborate kinds of behavior. The fitness functions, the things that are determining whether you survive or not, those are fairly coarse in biology. But let's imagine you have a coarse fitness function, like what's the overall lifetime of the pattern before it dies out, let's say, or how wide does the pattern get? You're not saying how it gets wide. You're not saying particularly, but you're saying how wide does it get. It turns out, with those kinds of coarse fitness functions, you can successfully achieve high fitness, but you achieve it in this very complicated way. You achieve it by sort of putting together these pieces of irreducible computation.
And that means that in the end, the answer I think to why biological evolution works is that it is the same story as what happens in physics and mathematics, actually. It is an interplay between underlying computational irreducibility and the computational boundedness of “observers” of that computation. So in the case of physics, the observers are us, doing experiments in the physical world. In the case of mathematics, it's mathematicians looking at the structure of mathematics. In the case of biology, the observer is kind of the environment. It's the fitness function. The fitness function is kind of the analog of the observer. And the fitness function is saying, you're a success if you achieve this kind of coarse objective. And the reason that biological evolution works is that there's so much power in the underlying irreducible computation that you're able to achieve many of these coarse fitness functions.
So if you imagine that the only way an organism could survive is if it breaks out of its egg and immediately it computes a thousand primes and does all kinds of other weird things. Right? It's not going to survive. I did that. Right. So I did it when I was born. As one does. But one isn't going to be able to hit that particular very complicated target. What actually happens is the fitness functions are much coarser than that, and that's why biological evolution has been able to work. But if you say, well, what's actually going on inside? What's going on inside is pieces of computational irreducibility being stuck together in a way that happens to achieve this coarse fitness function.
By the way, this is the same thing as what's going on in machine learning, I think. In machine learning, it's the same story that the fitness function in that case is you're trying to achieve some training objective. And you do that by sort of fitting together these lumps of irreducible computation. The analogy I've been using is it's kind of like building a stone wall. If you're doing precise engineering, you might build a wall by making precise bricks and putting them together in a very precise way. But the alternative is you can make a stone wall where you're just picking up random rocks off the ground and noticing, well, this one more or less fits in here. Let me stick that in that way, and so on. That's what's going on in machine learning. You're sort of building this stone wall. If you then say, well, why does this particular feature of this machine learning system work the way it does? It's because we happen to find that particular rock lying around on the ground or because we happen to go down this particular branch in the random numbers that we chose for the training. And so it's kind of an assembly of these random lumps of irreducible computation.
That's what we are, too. In biology, the history of biology on Earth, the dice has been rolled in particular ways. We have stuck together these sort of lumps of irreducible computation, and we make the organism that we are today. There are many other possible paths we could have taken, which would have also achieved a bunch of fitness objectives. But it was just a particular historical path that was taken.
One of the things that's kind of a slightly shocking thing to do is to take one of these evolved cellular automata that looks very elaborate, has all these mechanisms in it, has all these patches that do particular things and that fit together in these interesting ways and so on. And you ask an LLM, say, write a description of this pattern in the style of a biology textbook. And it's kind of shocking, because it sounds just like biology, because it's successfully describing, there's a, I don't know, it makes up, sometimes it can make up names for things. There's a distal triangle of this and so on, and interacting with this and this and this. And you go and you open a biology textbook, and it reads kind of just the same. It's a description, biology, and the detail in biology is a description of this particular sort of sequence of pieces of computational irreducibility that got put together by the history of life on Earth and that make us as we are today.
Now, since we care about us, it's very worthwhile to study that detailed history, that detailed lump of computational irreducibility that is us. But if you want to make a more general theory of biology, it better generalize beyond the details of us and the details of our particular history. And the thing that I've been doing actually most recently is I think we are at the beginnings of finding a way to talk about any system that was adaptively evolved. So in the case of the—if we look at all possible rules that could be going on in biology, many of those rules won't be ones that would have been found by adaptive evolution with coarse fitness functions.
So the thing we look at is what is biology doing? Biology is if nothing else a story of bulk orchestration of molecular processes. There are all these, one might have thought at some time in the past that biology is sort of just chemistry. And by that I mean in chemistry we sort of imagine we got liquids and so on and they just have molecules randomly bumping into each other. But that's not what biology mostly seems to be doing. Biology is mostly a story of detailed assemblies of molecules that orchestrate, this molecule hooks into this one and then does this, etc., etc., etc. And sort of discoveries in molecular biology keep on being about how orchestrated things are, not how this molecule randomly bumps into this other molecule. Right, it's sophisticated. Yes, but it also has this sort of mechanism to it. It's not just random collisions. This molecule is guided into doing this with this molecule and so on. It's a big tower of things, a bit like in these evolved cellular automata, which also do what they do through this big tower of detailed sort of applications of rules and so on.
But so, what I'm interested in is to have a theory of bulk orchestration. That's something that can tell one about what happens in any system that is sort of bulk orchestrated, which can include things like a microprocessor, let's say, which has its own sort of complicated set of things that it does. A microprocessor is not well described by the random motion of electrons. It's something different from that. But what is it? And does that theory that you make of the microprocessor depend on the details of the engineers who designed it? Or are there necessary features of any system that has been built to achieve certain coarse-grained purposes?
Hi everyone, hope you're enjoying today's episode. If you're hungry for deeper dives into physics, AI, consciousness, philosophy, along with my personal reflections, you'll find it all on my Substack. Subscribers get first access to new episodes, new posts as well, behind-the-scenes insights, and the chance to be a part of a thriving community of like-minded pilgrimers. By joining, you'll directly be supporting my work and helping keep these conversations at the cutting edge. So click the link on screen here. Hit subscribe, and let's keep pushing the boundaries of knowledge together. Thank you, and enjoy the show.
Just so you know, if you're listening, it's C-U-R-T-J-A-I-M-U-N-G-A-L.org. CurtJaimungal.org.