Transcription
I find a lot of discussion with philosophers frustrating. They say machines cannot understand something, and then you ask them what understanding is, and they don't understand. But understanding is, if we found a living system somewhere else in the universe, why are we so convinced we have to look for water?
The history of robotics will show you very quickly that that's not as easy as it looks when you try and take thinking away from the body. For Penrose, it is not enough for the brain to be quantum mechanical, and that is why he has pushed to a speculative, not yet known, quantum theory of gravity. Why? Because he needs that for his consciousness theory.
It seems to me this is the gap that's not present in all the other theories. I have to reply to that by asking you to consider this: water. An AI system is not made of water. We are. Why does that matter? If I take some pollen grains, grind them up very fine into the finest dust I can create, and drop them into the water, and then look at the water with a microscope, each of those grains is moving around like this.
Take the silicon chip; each of its molecules is completely fixed. It's like a crystal. It might vibrate a little bit, but it's not doing this. What that tells me is two things. First of all, down at the very deep molecular level, even sub-molecular, because water is the tiniest molecule you can have in the body, there is fantastic stochasticity.
Is there massive stochasticity in the nervous system? And the answer is yes. You stick an electrode, a tiny glass pipette, into my brain and record from any neurons there. You might take after the trillions of ends that are there. You will then, if you connect it up to a loudspeaker to sort of listen to the variations in the electrical potential, hear it crackling the whole time.
What you get is the voltage doing this all the time. There's massive stochasticity there. The AI machine does not have that; it can't have it. I challenged one of them and said, the next time you make an AI and claim that it has reached the threshold of becoming like us, can you try to make it out of water? The response was, really?
I said, you see, your difficulty is that evolution took two billion years to achieve what it's achieved in us and in other organisms. Do you think you're going to do that soon? No way. Now, my speculation, and I agree it's a speculation, is I do not think that AI will develop genuine consciousness for us to recognize that if it remains fixed with its molecules in the system that is enabling it to exist at all.
Because that's the physical basis of the AI. For as long as it's the case that it's made out of what is effectively a crystalline structure, it can't harness the randomness that we can do in all of ourselves. Incidentally, we are, what is it, 70% water? Something like that. I mean, to anybody who thinks that all of this is nice and solid, it isn't. It's mostly like that.
Yes, so if they are able to create a sort of AI system from water, do you think that that could be conscious? Well, you then would be creating life. What is the definition of life? It is molecules in a watery suspension surrounded by a lipid membrane, and it continues to go on indefinitely. That is the definition of a living system.
If we found a living system somewhere else in the universe, perhaps on Mars, why are we so convinced we have to look for water? That is because we know it's very unlikely that that will develop in any other way. So my main problem with these deflationary claims that people make, right, is that most of them prove too much.
Okay, because if I was going to accept that kind of argument, then why shouldn't I say, well, you know, you seem to understand, but I know that inside, really, you're just a bundle of neurons following the laws of physics? Like I've talked to AI skeptics, and I make that to me this blindingly obvious point to them, and somehow it goes completely past them.
Right? Like they don't even understand the point. Like they say, well, oh, but of course I understand. Like that's not the question. Like, but, but, but no, yes, it is the question. It's precisely the question now, right? Because the, you know, the question is if we accept that, you know, reductionism is true for us, that, you know, at one level we are bundles of neurons, or, you know, below that there's bundles of electrons and quarks following the standard model of elementary particles in a completely mechanistic way.
But at a higher level, we do have this understanding, this creativity, and so forth. Then it seems like we ought to say the same for an AI, right? That at the mechanistic level, it can just be fiddling ones and zeros. It can just be predicting the next token. It can be, you know, just doing a bunch of floating-point computations, you know, whatever deflationary language you want to use to describe it.
And yet at a higher level, it is thinking. It is understanding. It is learning. It is creative. It is all these terms that we apply to ourselves. If we're serious about looking for a distinction between humans and machines, which is not special pleading, right? Which is not just meat chauvinism, if you will, and which is not just pointing to something arbitrary, you know, that like we are based on carbon and they're based on silicon, right?
And, you know, without explaining why that matters, right? Then, you know, our options seem pretty limited. Right? And so, you know, one of the few people to take this seriously was Roger Penrose, right? Who said, well, you know, if I really wanted the brain to not be a computer, then, you know, I would have to pretty much throw away, you know, most of known physics and replace it by some speculative new thing.
And then incredibly, he bites that bullet. He says, fine, then that's what I'll do. I'll just speculate that the brain is sensitive to some new quantum uncomputable quantum gravity effects in the microtubules. Well, the story is a bit long and involved. In many ways, I was, when I was a graduate student, I had been worrying a bit about Gödel's theorem.
I'd heard about it, but I didn't quite like the idea because it seemed to say there were things in mathematics you couldn't prove, and I was a little bit worried by that. Anyway, so I went to this course, and what I learned was something very different.
If you have a system of rules which you could use to prove some theorem, and if those rules are such that you could put them on a computer, what Gödel does, as Steen described, and it's very clever, is how you can make a statement which, when you look at how it's constructed, it really can be interpreted as saying, I am not provable by those rules.
By virtue of your belief that the rules only give you truth, you can see that this is true. Nevertheless, it cannot be proved using the rules. By our understanding of what the rules are saying, it enables us to transcend the rules. This struck me as completely amazing that somehow the understanding that one has isn't following the rules.
There's something beyond that, and this transcending of these rules is a feature of our understanding, right? So it seems to me that whatever understanding is, and I don't know what it is, it's some quality that we have which enables us to transcend any set of rules that you have for proving things in mathematics, and they're not enough.
Yeah, but you can see by understanding them, you can go beyond them. Now, what does understanding involve? Well, it certainly involves consciousness in the normal usage of the words because one says you to understand. You couldn't imagine something understanding a thing if it wasn't even aware of it.
Yes, so being aware of something is a necessary ingredient. So then I began thinking, okay, well, what kind of, I'm a physicalist, so I believe what's going on in our heads is obeying the same laws of physics which is going on around us. There's nothing special injected into our heads which gives us a quality which is beyond the laws of physics.
I'm a physicalist. I believe it's the same physics in here as out there and everywhere. Yeah, different parts of it are emphasized in different ways. So I began to think about what are the laws of physics as we know them or knew them at the time I was thinking about this.
Okay, well, Newtonian mechanics, you can put that on a computer. Maxwell's electrodynamics, you can put all those things on the computer. I would think about Einstein's general relativity. I knew that was pretty hard at the time. They had got very far, but nowadays you can talk about inspiraling black holes, and it's really very, very detailed what people do.
Yeah, yeah, yeah. So sure, then what about quantum mechanics? Well, the Schrödinger equation, you can put that on a computer. It has a problem with how many variables you have to. Sometimes people regard it as a difficult problem for a complicated system, but it's not really fundamentally different.
So the Schrödinger equation also, but that's not the whole of quantum mechanics. No, quantum mechanics involves not just the evolution of the quantum state according to the Schrödinger equation or whatever you like to call it, unitary evolution. It involves measurement, and the measurement problem violates the Schrödinger equation.
You evolve a state, and it says if you made a measurement of such and such a type, it would give you a probability of that and a probability of this. If you evolve the measuring device as well as the system, then you get a contradiction because it says that the answer of the measurement is a superposition of this and that.
Truly, Schrödinger was worried about this with his cat. I mean, that's why he brought the cat into the discussion, right? To show that there's something wrong here. You see, quantum mechanics is incomplete, right? Right, because it doesn't explain the collapse of the wave function.
The Schrödinger equation is a smooth, continuous evolution of the state, but it's not what you get when you make a measurement. Yeah, when you make a measurement, which are sort of broadening the system a bit to include the measuring apparatus and all that, why doesn't it follow the Schrödinger equation?
It seems to me this is the gap that's not present in all the other theories. See, all the other theories, right up to the point, even including the Schrödinger equation or the evolution, unitary evolution in quantum mechanics, they are all things you could put on a computer.
So where is there something that you couldn't maybe put on a computer? And the argument I was making here is that it's in the collapse of the wave function, right? So there is some process, and I'm not taking the view, as many people did in the early days of quantum mechanics, that it's the conscious observer looking at the system and observing.
Yeah, that's the sort of terminology which suggests that observing the system, no, I don't believe that. That doesn't really work when you think about it in detail. I won't get into that, but it certainly doesn't really work. So I don't believe it's that.
However, it's a different role for consciousness, almost the opposite, that whatever consciousness is depends upon what this currently unknown, but we know something about it, which is to do with seeing how general relativity relates to quantum mechanics.
Right? But it's the other way around. It's not how you quantize general relativity, which is what many of my colleagues try and do, right? It's the opposite: how you gravitate quantum mechanics. You try to put the principle of equivalence, these great principles of Galileo, into the theory and the Einstein way of resolving these principles of Galileo into an overall theory and imposing that on quantum mechanics.
You can see when you look at this that the arguments tell you that with certain mass displacements, the superposition of it being one position or another has to go to one or the other in a certain time scale. Yeah, and that time scale you can estimate.
Yeah, and so this I claim is what's going on in the brain. Ah, and there, and where in the brain? I had no idea. I learned a bit of neurophysiology. I just couldn't see where in the brain it could be doing it.
Yeah, I rather sort of petered off in my "The Emperor's New Mind" without knowing the answer. But then Stuart Hameroff, who was an anesthesiologist in Tucson, Arizona, yeah, and he wrote to me and said, more or less, evidently you don't know about microtubules.
He didn't put it quite like that, yeah, but he said he suggested there's something. I mean, he was very interested in general anesthesia, and he realized that it's not a chemistry question. I mean, how you turn it off is a route into what it is, and what turns it off is not chemistry because you have many, many substances completely unrelated chemically which are general anesthetics.
One of the most striking being xenon, which is an inert gas or more or less inert. And so it's not chemical processes; it's some physical process. Well, we developed these ideas together, going off on different routes and trying to come back again all the time and formulating a scheme which is referred to as the Orch-OR proposal.
Yeah, yeah, yeah, yeah. Orch means orchestrated. It's not the O means objective reduction, so the reduction means the collapse of the wave function. Objective means it really is a physical process, and then there are curious features about when it happens.
For Penrose, it is not enough for the brain to be quantum mechanical, right? Because if it were merely quantum mechanical, then Penrose would say, well, the brain is still a computer. It's just merely a quantum computer instead of a classical one, right? But that's in some sense just as mechanistic.
I mean, it's true that probabilities would then be involved, but you know the whole point of quantum mechanics is that it gives you a precise algorithm for calculating the probabilities. And so if we were just, if we were merely probabilistic in the way of a roulette wheel, you know, that wouldn't seem like any kind of free will that was worth having, right?
And so Penrose specifically, as you said, he wants something non-computable in the brain because he believes, because of an argument that I think is fallacious, but you know, he believes that the brain has to be uncomputable in the Turing sense.
And even quantum mechanics would still give rise to a computable description of the brain, and so that is why he has pushed to a speculative, not yet known quantum theory of gravity, right? That he believes, you know, unlike most physicists, ought to involve some non-computable phenomena.
Right? Why? Because he needs that for his consciousness theory, right? I'm trying to be much more conservative, ironically, right? If I want to have a fundamental difference between, you know, humans and digital computers that is grounded in currently known physics, then there seems like only one that I can think of.
And that is that digital computers, as we have created them, right, run programs that are just pure classical information that can be copied, okay? That can be backed up, that can always be rewound to an earlier state, right?
And so these are always capabilities that we have with any AI, right? If you don't like the answer that ChatGPT gives you, right, you can always just feed it the same prompt, right? Just like try again from the same initial condition and see what it would do differently, right?
If you feel like, you know, GPT has now gotten a bad impression of you from your conversation, you can wipe the slate clean, right? You know, like I'm sure so many people wish that they could have done that with other people, maybe when they're on a date or something.
Right? But you know, we can't. Actually, just on the flight on the way here, I watched a movie that was based on that premise. You know, someone had about time, someone has the magic power to just go backwards and try a conversation again if it went the wrong way, right?
We can't do that with people, right? People, you know, seem to be running on this noisy analog hardware, okay? Which is not, you know, copyable, at least with any currently known technology, which is not backup-able, which is not rewindable to earlier states.
You know, and that gives us a kind of ephemerality, right? That AI's don't have. Now, you might say that, you know, this is not an advantage to our side, right? This merely makes us more frail, right? More limited compared to the machines.
And so that seems like cold comfort, but you know, in a way, it also makes our choices more meaningful, right? Because we only get one to make them, and there's no going back, right? With an AI that, you know, you could be running in, you know, a thousand different copies, or you could be trying again and again from the same initial condition.
You could say, you know, there's, if you ask GPT to write a poem, right, it's never really committed to anything because if you don't like that poem, it could generate 50 more from the same probability distribution, right? Whereas Shakespeare, right, we're never going to get Shakespeare to try again at writing his plays, right?
The ones he wrote are the ones he wrote. So it gives human decisions, you know, a kind of permanence or significance to them that I think AI, just because of their clone-ability and their rewind-ability, are never really going to have.
And that's certainly a difference that's going to be very important in practice. Now, is it more than that? Is it, you know, is this actually a hint of a more metaphysical difference or of a deeper, you know, difference? Like should we speculate that consciousness can only inhere in systems that are fundamentally unclonable?
Let's say, you know, I'm willing to entertain that speculation. I don't know whether it's true or not, but you know, it makes at least more sense to me than a lot of other speculations I've heard about the physical basis of consciousness.
Roger Penrose's book demonstrates that Roger Penrose doesn't understand what consciousness is, and he would also suggest that he doesn't. Right? That's the thing that's so puzzling to him, and many of our best minds don't understand stuff, and the others also don't understand stuff.
So maybe the goal is not human understanding. Humans really are very, very bad at understanding things. Humans interact with reality without understanding how they interact with it. It's very rare that we deeply actually understand something.
So instead, what we are producing is coherent behavior, and our AI are also producing coherent behavior. And I think that understanding requires that you build a world model that is completely coherent and that is grounded both in first principles and in observation, right?
And something that is outside of the realm of what human minds can do, if you're honest about it, right? So I think AI is our only chance to build a system that is able to understand something. And philosophers have understood this for a long time, like since Leibniz, Frege, Wittgenstein, and so on.
People have tried to mechanize the mind, to turn it into a mathematical principle, to get to a system that is able to bridge between mathematics, where we have truth and falsehood defined, and philosophy, where we talk about the actual world and the sphere of our ideas and the possibilities of what could exist.
Right? Human minds are not deep enough to make that bridge, but if you can build something that is like a human mind but can be scaled up, we might be able to build something that actually understands.
People complain to me, say, well, you're only talking about a very limited thing. Of course, it is, but the point is if what I'm saying has relevance to our understanding, it may not be relevance to all features of understanding. Sure, I'm quite happy with that.
But nevertheless, it seems to indicate there's something outside computation in human understanding. The endeavor of artificial intelligence, as a sort of historical movement, has attempted to have an idea of what artificial intelligence is, and with that comes a lot of fantasies around the sort of holistic idea of thought, which actually is quite a very inhuman idea of thought.
It's an idea that abstracts thinking from the body and that has this sort of idea of intelligence being something that can be improved exponentially, that can get faster, that can get more efficient. And of course, the history of robotics will show you very quickly that that's not as easy as it looks when you try and take thinking away from the body.
The more that we try and reify the concept of artificial intelligence, the more problems we find with how it's actually related to the body, with speech, with also a lack of knowledge, a lack of intelligence, what we could call stupidity.
I think that the mind is a pattern in the physical reality. It's a causal pattern that is self-reinforcing, and we know these causal patterns in other contexts. We call them software. The software is disembodied, but it's still interacting with physics and the large parts of the physical world that we cannot explain if we do not assume the existence of software.
If you look at your phone, for instance, the software that is running on it doesn't really care about how many transistors are there and how much memory you have in there. But it is implemented on the activation states of the transistors, and so there's this interaction between hardware and software where the hardware is providing the mechanism and the software is the pattern that actually matters.
The actual invariance that we care about is consciousness. I can't, it's not as physical as other software is, but the consciousness that we have, there's a big difference between the software that you are writing on computers. It's self-organizing. It needs to be able to convince the cells to run it.
And the cells are little neurons in your brain, little single-cell animals that try to survive and serve their own interests. And if you want to run on them, you need to convince them to run you. So you need to provide some benefit, and you are some software that is basically possessing groups of neurons.
And you do this by training the neurons to run you. So there needs to be some kind of seat that enables this kind of entraining of yourself, something that is able to colonize the brain, turns it into a mind. And inside of that mind, which is like a protocol, there generates a game engine that is giving you a simulation of a physical universe in which you sound, colors, people's emotions, their interactions, a model of yourself.
And at some point, this conscious perspective is relocating into this nexus of the personal self, and you think of yourself as a human being. You use "I" to refer to yourself, and this conscious creator of your mental contents that is dreaming you thinks that it's no longer the dreamer, but it's a person inside of that dream.
And this is our normal everyday experience. So the brain is hardware; consciousness is software. I'm struggling with the analogy somewhat, and it's not an analogy; it's literally correct. Okay, so it's software. I'm struggling.
With software, but software is basically, if you think what software is, if you are interacting with computers, it is a disembodied causal pattern, right? It's very much a spirit. Only the software that we are building is not self-organizing; it's constructed by humans, and it works because we force the hardware to enact it.
We make it so deterministic that the hardware has no choice, but your cells at some level have a choice in the same way as a group of people has the choice to run a religion or not. And so you need to convince all the individuals to believe in the shared spirit of an organization, of a nation-state, of a family, of a relationship, or of a religion.
And in the same way, you need to convince yourselves to believe in you. And I think that the human mind itself is limited in our ability to understand because it's quite mushy, right? We have this vague reference, and they are grounded in perceptual constructs that we largely only know as a pattern matching and not as a deep understanding.
So I find a lot of discussions with philosophers frustrating who say machines cannot understand something, and then you ask them what understanding is, and they don't understand what understanding is. They just have this very vague pattern matching, and this pattern matching is not good enough to tell them that the machines understand something or don't.
The uses of AI for instrumentalizing language is kind of an inevitable part of, as these types of technologies become more sophisticated. I think it's something that we should have envisaged. I think it was quite clear that that was going to happen.
And I think, you know, these lessons are already within psychoanalysis in the sense that this idea of the robotic nature of the symbolic, of language, of systems of meaning, which human beings sort of are born into, that actually aren't particularly human. They're systems that we inhabit, but we don't really choose them.
So they're like machines. You know, when you're born, you're born into a language. You're born into a system of codes and laws, and you, to a certain extent, cannot decide what those are going to be, and you have to mold yourself around them.
And that sort of like bodily, physical experience of being a human being smashed into a system of symbols produces the human being who goes about its sort of daily life obeying laws and speaking a language. And so, of course, when you take forms of that process, like language, and you sort of put them into an exponential system, and then you get at the end of it something like ChatGPT, which of course can extract linguistic systems into all different kinds of directions.
You know, you immediately feel like, oh my God, well, where's intelligence going? It's going away from humans, and we're not able to keep hold of it anymore. I sort of feel like that was already happening. That's just a natural part of the sliding of the signifier that already starts happening with the abstraction of language in the first place.
We are interested in having a system that works very much in the same way as we do, where we do, that is able to understand the world, that is able to relate to itself, to the environment around it, that can deeply interface with us and that can probably also scale up.
Because if it has those abilities, it's not constrained by the size of its brain or by the processing speed comparable to the human nervous system. So it's not really conceivable that it's going to stay at the human level or be very human-like for very long.
And do you think that this is inevitable, this direction? I think it's inevitable, but I think we should be wary of it, and I think we should be much more in control of our technologies. I think that we need to be much more aware of how they function.
I think there needs to be more thinking, hard thinking, philosophical thinking, psychoanalytic thinking in artificial intelligence, and it shouldn't be just left to people with huge amounts of money to make these decisions for us.