Don’t Believe AI Hype, This is Where it’s Actually Headed | Oxford’s Michael Wooldridge | AI History

The Singularity is why is that? There is this narrative out there, and it’s a very popular narrative and it’s very compelling, which is that at some point machines are going to become as intelligent as human beings and then they can apply their intelligence to making themselves even smarter. The story is that it all spirals out of our control, and of course, this is the plot of quite a lot of science fiction movies, notably Terminator. I love those movies just as much as anybody does, um, but it it’s it’s deeply implausible, and I became frustrated with that narrative for all sorts of reasons. One of which is that that narrative whenever it comes up in sort of serious debate about where AI is going and what the risks are—you know there are real risks associated with AI—it tends to suck all the oxygen out of the room, in the phrase that my colleague used, and it tends to dominate the conversation and distract us from things that we should really be talking about.

Right. In fact, there’s a discipline that’s come out of this called existential risk. Right? It’s kind of the worrying about the Terminator situation and figuring out how can we perhaps better align these super-intelligent agents to human interests. Um, and if you look at not just the narrative but actually the funding and what the smartest people are devoting their time into thinking in, in not only companies but policy groups, X-risk, existential risk, is the dominant share of the entire market, so to speak.

Why do you think this narrative has gained such a such a big uh big following? I think it’s the low probability but very very high-risk argument that I mean I think most people accept that this is not tremendously plausible, but if it did happen it would be the worst thing ever. And so very very very high risk, and when you multiply that probability by the risk, then it’s the argument is that it’s something that you should that you should start to think about. But um this when the success of large language models became apparent and ChatGPT was released and everybody got very excited about this last year, the kind of debate around this sort of reached sort of slightly hysterical levels, um, and it became slightly unhinged at some point. My sense is the debate is calmed down a little bit and is being more focused on the actualities of where we are and what the risks are.

Right. I think that’s quite a charitable reading I think of the psychology, right? It’s a rational calculus; there’s small probability, but there’s a large sort of cost. I study religious history, and when I talk to people in the exis world, the psychology kind of reminds me of the Christian apocalyptic; that there’s these people throughout Christian history that are like “Now’s the Time,” you know. This happened most recently probably when we were uh going through the Millennium, right? 1999. And it’s this psychological drive that wants to grab at something total and eschatological in a way to orient the entire world. So so people, I guess what I’m trying to highlighting is maybe you can see some of the psychology and climate risk as well. It’s not to say that these things aren’t true, right? It’s not to say that the world isn’t ending in Christianity, the climate isn’t changing, or there is no exis; it’s that the reason that people seem attracted to this narrative is almost a religious phenomena.

I think that’s right, and I think it appeals to something almost primal and kind of human nature. I mean, it’s most fundamental level it’s the idea that you create something, you have a child, and they turn on you, you know, that kind of the ultimate nightmare for parents. You know, you give birth, you nurture something, you you create something. Exactly so, uh, or, you know, and this that narrative that story is very very resonant. And for example, you go back to the the original science fiction text Frankenstein; that literally is the plot of Frankenstein: you use science to create life, to to give life to something, to create something, and then it turns on you, and you’ve lost control of that thing. So it’s a very very resonant idea I think, and so very easy for people to latch onto.

Right. You know, it’s easy for us to critique the psychology here, but but what’s what what do you think is wrong or what do you think people mis- about the argument itself—that once we have super-intelligent or or at least uh on par with human-level uh machine intelligence that they can recursively improve upon themselves? What what do you think people are missing when they give too much weight to that that argument? The frustrating part is the Skynet part of the argument, you know, the kind of the Terminator thing—that suddenly this will spiral out of control in ways that we just can’t control in an incredibly short um an incredibly short period of time. If you look under the hood of of how these things work and how many patches are required to hold AI together, um, it just it just doesn’t seem terribly plausible.

More concretely, there are basically two arguments for how existential risk might come around, and the first is the famous paperclip argument, which I’m sure you’re familiar with. You know, so you build a highly intelligent machine and you ask it to build as many paperclips as possible, and it follows your instructions in ways that you didn’t anticipate. Right? For example, enslaving all of humanity to build of humanity and turning them to the production of paperclips, uh, you know, until it turns everything into paperclips. That’s the uh that’s the paperclip argument, and there is some strength to that argument in the sense that AI can go wrong in those ways, but for it to uh to to hurt us it has to be empowered to hurt us; we have to give it the keys; we have to give it control. Assuming there’s no guardrails—and there’s no guardrails—and again that just doesn’t seem terribly plausible that that we would do that. I mean, it would be a dumb thing for us to do to hand over the nukes to an AI. So that’s the first argument about how AI might become an existential threat. The second argument is just that we build very very intelligent machines which develop their own goals which aren’t aligned with ours. Now this is much more nebulous; we don’t know how that might happen, um uh it’s and and so it’s slightly harder to address, but we really aren’t at the moment anywhere near that, and I don’t see even with very very powerful AI that we have have now the roadmap from where we go to that.

So your understanding, the first case is when the AI is executing our goals but not ingesting the kind of uh assumptions and implicit values the humans are imputing, whereas the second one is a have developed their own goals, um, and there it seems even more far-fetched because LLMs, as as powerful as they are, they don’t seem to be have any semblance of agency, right? And that’s fundamentally what it would require. So me and my friends have actually come up with this half-joking term called existential risk risk, which is the risk upon a society that focuses too much on existential risk and away from other other risks that we could actually be facing due to AI today. If you can wave a magical wand and and swing the narrative of AI away from X-risk, what are the actual problems uh and conversations that we should be having right now? We are heading into a world where basically within a decade, two decades, think at the most, um, pretty much everything we read uh and see on social media and the internet is going to be AI generated, and we’re not going to know what’s real and what isn’t real in that world. And there are many many risks associated with that—that society just fragments because there is no common core of beliefs anymore—that we’re all obsessed with some particular issue and that social media and the internet is just driving us around that one particular issue because AI is programmed to pick up on the issues that you care about and to feed you stories emphasizing those risks and so on. Where I was particularly concerned is going into elections in the US, the UK; I was really worried that what we were going to be see was social media drowning in AI-generated fake news. We didn’t see that as it happens, at least not on the scale that I feared it might occur, um, but nevertheless I wouldn’t take my eye off that as a risk. I think that’s a very very real risk, um, that uh that uh autocratic states uh control media just use AI to generate stories, endless stories, fake news stories, that populist politicians do the same thing and so on, and that we just drown in fake news till we no longer know how to tell what’s real and what isn’t and don’t trust anything as a consequence.

Given all these problems, let me read you uh a quote from your book: “Do we need laws, maybe even international treaties to control the development of AI in the same way that we do for nuclear power?” I find the idea of introducing general laws to govern the use of AI rather implausible; it seems a bit like trying to introduce legislation to govern the use of mathematics. What do you think should be the role of government and policy, if any, in the mitigation of these risks? So what I’m concerned about is some some sort of naive attempt to create a neural network law, you know, “Thou shalt not use neural networks,” or something like that, um, and that’s what seems to me to be implausible because neural networks under the hood are just a bit of mathematics—actually not terribly complex mathematics; there’s a lot of it, but it’s not terribly complex—and so regulating that—well, where do you draw the line? I mean, is is a bit of basic statistics, you know, the kind of thing that you would routinely do, is that AI? Um, neural networks are quite a lot of linear algebra; do we outlaw linear algebra when we when we write programs? So trying to regulate technology by pointing at neural networks and saying, you know, we should not use these is is is problematic in a way that I think pointing at nuclear weapons, you know, and nuclear fission devices is not. We can easily identify a nuclear fission device; I don’t think there’s much debate about that, uh, or the use of chemical weapons and so on—outlawing the use of uh chlorine gas or whatever in in weapons, you know, these things are fairly easy and robustly identifiable. AI isn’t, and it’s a really gray area about whether something actually is AI or isn’t AI. So my preference would be that we focused on the uses of technology, and I think the one that I pick up on in the book is surveillance technologies. If somebody’s using surveillance technology on me, I don’t care whether it’s a neural network or a logic program or guy sitting in a in a cabin watching. Exactly what I care about is somebody is using surveillance technology on me, and that’s where um uh the outlawing should happen. And so what I would prefer when we look at regulation, rather than aiming for some general neural network law, is to look for specific sectors—law, health, defense and security, uh all of those different areas, finance, um uh all of those different areas, education and so on—and think about what are the issues that AI raises there about the use of the technology and legislate around those.

Right. And maybe we can tie this into your critique of X-risk as well, because I think a lot of the impulse or the intuition behind this these like neural net laws as you described is kind of preemptive—like, “Oh my God, like if we don’t set up the right controls like it’s going to spiral out of control and then we’re going to have Skynet”—whereas you’re saying because we should be more uh moderate in our expectations of what it can and cannot do, including the harm it can do, we can just regulate this like anything in any technology or any physical industry and infrastructure that we had in the specific use cases. That’s what I would that’s what I prefer to see.

So far we we’ve talked about the forward-looking view. I I want to spend the rest of the interview really diving into your book and talk about the the historical view of of AI, but let’s begin with a question that I imagine many people in technical disciplines would would have, which is: why should we care about the history of AI? Because the history of AI still has lessons to teach us, and one of the big lessons that it teaches us is that it’s very easy to get overexcited and to read too much into what you’re seeing in AI, and people have done that on multiple occasions in the past. Um, now I think with the current wave of AI, I think there is real substance here; I think we are at a breakthrough moment, uh, but I’m not convinced that we’re at the end of the road in AI or that the Transformer is the magic ingredient. A few years ago people were saying deep learning alone is the magic ingredient for AI; now it’s the Transformer architecture and so on. I don’t think either of those things are the magic ingredient; I think there are some ingredients that we don’t yet know about. And I think your point about there’s things for even serious AI researchers to learn from the history of AI can be made even stronger: it’s not just the negative of oh you know uh calm your calm your excitement, but let me draw an analogy. Um, in philosophy there’s an idea that a lot of moral intuitions, good moral intuitions, are lost through paradigm shifts. So we gain things in this whole Christian worldview, but we moved away from the Roman world and the Pagan world, and there’s things to be rescued from that world that have been forgotten. And my my training, early training, was in STEM; STEM usually doesn’t study historical stuff, right? You usually just study the latest physical theories, but there is a view of even STEM innovation, Thomas being the biggest proponent, as being these paradigm shifts—that there are things that are important that are lost in previous paradigms—and I think we’re going to see this today. And so the positive pitch I would say to give even to serious technical researchers to the history of AI is that there are methods and ways of thinking about programming artificial intelligence in general that have been overlooked in our current paradigm that perhaps might be rescued and is perhaps perhaps what we need to get us to the to the next frontier.

I think that’s right; I think we are, to use Kuhn’s phrase, we are at a paradigm shift moment; there’s pre-GPT and post-GPT. It’s been boiling up for a decade or more, really. I mean, things became clear that things were happening in neural networks around about 2005 with the advent of deep learning. So here are the key history points: 2005, advent of deep learning; 2012, people realized GPUs could be used for training neural networks, and all of a sudden you got 10 times more bang for your for your buck uh by using GPUs in terms of training neural networks, and so you can multiply what you do and stuff gets really overheated; then then in 2017 there’s a Transformer architecture; 2020 there’s GPT-3. Those are the kind of moments, but we are in a paradigm shift right now, and in computing I genuinely believe that the world is shifting now from a kind of an era where we were very interested in coding exact and optimal algorithms and thinking what is the right algorithm for solving this problem; now it’s “give us the data, we’ll just throw it at machine learning and let machine learning sort it out.” Do we care about exactly how it’s doing it? Not necessarily; it’s just going to give us the answer, but it’s so many cases it turns out that the answer it’s giving us is an incredibly useful one, even though we sacrifice something, and what we sacrifice is kind of guarantees of correctness and optimality, but nevertheless it just turns into such a powerful tool. So the the the paradigm shift is towards a kind of data-driven world.

Totally. And and um so I did my uh computer science degree 2016 to 2020, and uh if you had asked me the history of AI, I would have given exactly what you told me and no more, but reading your book it extends greater another at least half century, right? And in fact there’s a whole other dimension of AI about explicit programming—what you call symbolic AI—that’s been completely overlooked; that was the dominant paradigm over ML, but now when we think AI we think ML. And so there are perhaps intuitions that we can rescue in the next hour for even the leading technical researchers, but let’s go to the very beginning with Alan Turing. Tell us about how he laid the groundwork not just for AI but for computer science as a field with his Turing machine. Yeah, so Alan Turing, for all that he’s now one of the most famous mathematicians I think in history, there’s still a huge part of the Turing story that people don’t really understand. And what everybody knows Turing for is the code-breaking work he did at Bletchley Park, which was an incredibly important part in the in the Allied victories, and that’s led to the Hollywood movie and so on. I mean, it’s an entertaining movie, but but hopelessly inaccurate. Um, but anyway, go back to the 1930s; he’s doing his PhD um in uh in Cambridge, and there’s one of the big mathematical problems of the age—the Entscheidungsproblem; it’s called the translates as the decision problem—and roughly speaking what the decision problem asks is: can you automate mathematics? Can you reduce mathematics to just a procedure that you follow—that is, take away all human insight—just to a procedure? And it was one of the defining problems of the early part of the 20th century, and with incredible precociousness I think Turing set himself the task of attacking the Entscheidungsproblem and solved it very very quickly. But to solve it, he invented a kind of mathematical machine, a machine that follows instructions, and at the beginning it was just a mathematical abstraction, but his work on code-breaking machines in the Second World War leads him and a bunch of other people to realize that actually you could build these Turing machines, and a Turing machine basically with a few practical tweaks is the modern digital computer. That’s all a computer—that is, a machine for following instructions. So it’s kind of one of the great ironies of mathematical history that computers get invented as a byproduct. I mean, he wasn’t setting out to invent machines that could do things; he was setting out to solve the Entscheidungsproblem, and he had to invent computers in order to do that. But he did that. After the war he goes and works on the first computers, right? And those computers, those very early incredibly crude computers, there’s there’s there’s less than a handful in the whole world, but they’re capable of what seem like incredible intellectual feats; they can do huge quantities of mathematics very quickly and very accurately—much more quickly and accurately than any human being could do—and people start to think, “Are these machines intelligent?” And that puts the idea of AI in the air. And what I think is amazing about that period is we went from the beginning of the 1950s where there were probably two or three computers in the whole world—ridiculously crude by today’s standards—by the end of that decade we’d gone to having machines that could do the rudiments of planning, problem solving, playing a decent game of chess or checkers. You know, from having nothing whatsoever to machines that could do those things—extraordinary progress in just a decade. There are giants, intellectual giants, where even their kind of uh secondary thoughts end up spawning entire disciplines, like Newton, calculus, for example. What I want to emphasize about the Turing machine, especially for our non-technical audience, is that it’s what you can tell it to do is very rudimentary; it’s literally unambiguous, explicit instructions—go like “if A then B, move pointer from E to Z,” right? Like “write this value at this place, read this value from that place”—extremely rudimentary procedures. And I want to emphasize again for our non-technical audience that is still the foundation of our current computers today; it’s not like we’ve designed a fundamental new paradigm. But that’s what’s fascinating, yeah, is that we can get from those again deterministic simple explicit procedures to the kind of emergent behavior from of of ChatGPT. And why this is so exciting for me as a philosopher was I remember in my undergrad uh uh I was debating with a Kantian scholar, uh you know the Kantians, you know, famously or infamously, you ask believe in free will, and the Kantian scholar said, “Well, no deterministic system can come up with the kind of not even theoretical reasoning but the common sense reasoning you and I can do.” And you know, I was I was in ML class, I remember very very fondly, 4771, and I was learning how to use explicitly only deterministic systems to come up with the kind of common sensical natural language manipulation that humans do. And and and this is just one example of how, as you said, philosophy is becoming experimental science.

Yeah, no, and I completely agree with you. I think it’s one way of framing the AI question is: can intelligence be reduced down to those incredibly simple instructions, explicit instructions that computers—and if you’ve never done any programming it’s quite hard to imagine how dumb simple instructions—this is why computer programmers are paid a lot of money, right? Because you know you require a special mindset to be able to think down at that level and to understand how machines operate. But yeah, uh, and and it is remarkable that what we see in um, you know, large language models, state-of-the-art AI, uh these kind of very dazzling capabilities ultimately are just reducing down to those very very simple instructions. But my golly, there’s a lot of those simple instructions in order to do what they’re doing. I can’t help but compare our current moment today with two watershed moments in maternity: the Copernican revolution—“humans don’t think you’re too special in a universe”—Darwin—“humans don’t think you’re so special in an animal kingdom.” It feels as if our last bastion, our our last sacred ability, a sacred thing that we have in our possession—our intelligence—has been reduced down to binary bits.

Yeah, I don’t think we’re all the way there yet. I think uh you know I think the fundamental nature of of human beings—I mean one of the fundamental components of human beings is that we have experiences; we experience the world. Um, that’s you know nobody really understands what consciousness is, but roughly speaking people agree agree that that that ability to experience things from a personal perspective, and that your personal perspective is private and unique to you, and I can imagine what you’re experiencing, but it really is private and unique to you, uh, and you know, are we at the point of getting that from machines? I think no, definitely not. And and how we might do that is is very opaque to me, if it was an interesting thing to do at all, right. Um, and this is a perfect segue to talk about the other thing about Turing he’s famous for, which is the Turing test. Turing test. So so tell us about why he formulated this test and how it impacted the development of AI. So in the 1950s then we have the first digital computers, the first computers that operate according to the structures that we recognize today, and they came out of Turing’s envision for the Turing machines, and then people realizing that actually we could build machines that look like this. So we’ve got the early 1950s, we’ve got the first digital computers, um, and this starts a debate, a kind of public debate about AI, even though it isn’t given that that that term. And Turing gets frustrated because people dogmatically insist that computers will never be able to do X, where X is creativity or emotion or whatever, uh, and he—and crucially—machines will never be able to understand something in the same way that a human being is. So he invents the Turing test, and the very famous Turing test is beautiful in its simplicity, uh, and it’s a test for or indistinguishability. So the Turing test goes as follows, roughly speaking: you have uh a human judge who’s interacting with something via—as he described—a teletype, but you know imagine a computer screen where you’re just typing uh you know typing whatever you want; they could be questions, but they could just be whatever you want, and you’re getting responses through that screen, and you don’t know whether the thing on the other end that’s producing those responses is a human being or a computer program. And the Turing test says: if you cannot reliably tell the difference—that is, if this machine can effectively pass itself off as a human being—then stop arguing about it; there’s no point in arguing about it because you cannot distinguish between what the machine is doing or what a human does by any reasonable test. So there’s two ways to interpret the Turing test philosophically: one way is to reduce metaphysics to phenomenology, and this is to say, “Look, the metaphysical question of ‘does it understand something? Is it really thinking?’ is totally collapsible to the phenomenological, the empirical question, ‘can we distinguish the outputs?’” But or it could be making an epistemic point, which is to say the metaphysical question doesn’t really matter; let’s just focus on the empirical question. Which reading do you think Turing’s given there?

Yeah, not clear. I think which which reading I suspect the former. I think he probably just thought was I suspect the former. I think he probably just would thought we should there’s no point in having the debate after this point. I mean, do do you agree with that premise? I mean, surely surely surely not, right? I mean, what this reminds me of actually is the behaviorist, the psychological school, and the behaviorists, even in the most charitable interpretation, think that everything about a human uh uh can be known through their behaviors and interactions with the external environment. That’s the most terrible reading; the least charitable reading is—and and there’s literally passages they almost literally say this—“the mind doesn’t exist; the consciousness doesn’t really…” I I don’t even know like what that could even mean for it to be plausible, but but but that is almost kind of the mistake that I see Turing making here. So but I think what this illustrates to us is that the Turing test, beautiful as it is, it’s not a terribly interesting test for intelligence in human beings. I me I think what Turing said is it was kind of just frustrated with the debate and said, “There’s no point in having this argument. I mean, if it is just doing something uh that that is indistinguishable, then why are we even debating after that point?” Uh, and in a purely practical sense, whether it’s really um really understanding in a way that human beings are is kind of irrelevant at that point. There are distinctions between strong and weak AI, and the idea of strong AI is that we what what what we have what we’re aiming for or what we have is machines that really understand and experience and so on in the same way that a human being or animal does. Um, the weak version of AI is: “No, they don’t really understand, but they can simulate those things.” I’m not terribly interested in strong AI except after a couple of glasses of wine in a in a in a in a chat with colleagues, and I don’t know very many AI researchers that really are interested in strong AI. The goals of AI are much more pragmatic. By the way, I think for all practical intents and purposes we passed the Turing test at some point in the last few years, um uh but what that illustrates is I think is just actually the limited value the Turing test has as a real test for uh for intelligence. Right? For me, the only reason the consciousness of a uh a computing machine uh has or does not have um the only real concern for me is is whether we have to treat them as moral agents. Right? If you think that a a machine might be suffering, it doesn’t want you to turn it off, we might have to give some weight to that, but that seems to be like the only possible reason why someone would be interested in in strong versus weak.

Right. Yeah, I think people, some people just think it’s a it’s an interesting thing. Yeah. Um yeah uh the idea of you AI as moral agents, I’m worried about this. Um there is a uh there is a body of work which is all about trying to equip AI with kind of ethical and moral reasoning, um and I understand why people want to do that, so that we have machines that make choices that we would want them to make. What worries me about that is that it allows people to try to abdicate their moral and ethical responsibilities—“Wasn’t my fault; it was the machine’s fault,” you know. Um, but we can’t hold a machine to account for its actions in the way that that we can hold a human being to account, um, and I say I’m really concerned particularly in like the military sphere that what we’re going to hear is “Wasn’t me; it wasn’t me; you know, it wasn’t our fault; it was the AI did it; the AI chose the target—that school—and fired the missile; it wasn’t our fault at all.” And so I think what I want is not moral AI; I think it’s moral human beings, and it’s the people that build and deploy the AI where the responsibility and the ethical considerations have to sit, and they are the ones that we need to hold to account for the actions of the machines that they deploy.

I see. So that’s Turing, and that’s the beginning of the field of not just artificial intelligence but computer science as a whole. I want to move on to the actual history of the implementations, and I’ll begin with a quote from your book: “Historically, AI has adopted one of two main approaches to this problem. Put crudely, the first possibility involves trying to model the mind; the alternative is to model the brain. To model the mind is what we’ve been talking about as symbolic AI—to give it explicit instructions of what to do, to to model the the processes that we rationally consciously go through in our heads. To model the brain—that’s machine learning; that’s the neural nets; that’s to model the architecture, the physical architecture of the brain, even if we don’t have that much insight into what is actually going on.” Let’s talk about symbolic AI first; let’s talk about modeling the Mind first. The golden age, as you described it, 1956-1974; tell us about this first boom of AI. Yeah, by the end of the 1950s we’ve got machines that can show the rudiments of intelligence—that can that can plan, that can do mathematics—which, to be frank, you know, would be above the typical level of the people on the street, you know, here in Oxford, you know, your chances of getting somebody who could uh who could tell you what Goldbach’s conjecture was or something like that um would would be limited. So you’ve got machines that can do mathematics, that can that can solve problems, play games, and so there is this real excitement that you know actually we’re going to be very quickly making progress towards something like full general intelligence, and it’s called the Golden Age because you know we went from having nothing to having machines that could do those things, um and there was a period where you know where you know the the the modus operandi for a for a PhD student was in in AI was: “Well, let’s think of some task that requires intelligence in in humans and just build a machine to do those things.” And turned out to do crude versions of those things were turned out not to be that that hard, but there was this massive optimism for that reason—that progress was just going to be swift; people thought within decades that they were going to be at the end of the road in AI; we’d have full general intelligence. AI hype is not a new phenomenon; AI hype is very much not a new phenomenon, and uh and uh by the early 1970s it becomes clear really that progress is stalled, and there are lots of reasons why progress stalled. One of the reasons that progress stalled is people were looking at artificial versions of problems rather than real problems; that is, they were looking at some problem in the real world, like a robotics problem, and then coming up with a simplified simulation of that problem in a computer; they were able to solve it in the simple simulated version, but that simulated version didn’t address any of the problems that were there in the real world problem. Um, so classically in robotics people would do simulations of robots in warehouses, and you’d look at a screen and you’d see a simulated robot carrying packages around, and it looks very compelling—you know, “Great, okay, so show me the system in the real world”—but robots carrying a package round in a in a warehouse in the real world is nothing like the simulated version. And so those simulated versions, they were called microworlds, uh, and again a standard modus operandi for a PhD student is: “Come up with a microworld for your particular problem, uh whatever it was, show that you could build a program that could solve it in that microworld,” but actually you’ve you’ve abstracted away everything that’s difficult about the real problem in the real world, and you know and and research funders would say, “Fine, show us this then in a real uh warehouse.” They wouldn’t be able… You described the general philosophy and strategy in this period of the golden age as “divide and conquer.” So this is splitting out what we conceive our mental faculties to be and then trying to build—I mean mostly search algorithms in different variations—to satisfy that. So one example would be the the towers of uh towers of Hanoi, exactly, where you have to move the the the disks on top of one another to get it to the right shape. And those are the traveling salesman, right? You give given given a list of cities and try to find the the optimal algorithm there. And those are the type of problems that people seem to be working on. There was a common ceiling that people were hitting across these problems, um, and that had to do with NP-completeness. So can you give our our lay non-technical audience a rough idea of what this means? So a lot of approaches to AI in uh in the early days involved something called search, and search just means if you’re given a particular problem, just look through all possible candidate solutions. So uh we mentioned the idea of the traveling salesman problem; the the traveling salesman problem—you’re given a particular map and that the salesman so to speak has to visit a whole bunch of cities on this map and return to return to base—can the salesman do that on a certain budget of fuel? Um, that’s the traveling salesman problem, and so one way to approach that is just to look through all the possible candidate solutions. The problem is that the number of candidate solutions in that case just grows astronomically. Um, so for example, if there are something like 70 cities, there would be more possible candidate solutions than there are atoms in the universe; you will never have a computer that could exhaustively look through all of those candidate solutions. And it was assumed in the early days that we would be able to fix that problem, uh it’s called combinatorial explosion, uh we would be able to fix that problem somehow—that we would find some techniques to be able to do it. By the early 1970s there was an emerging theory of what’s called computational complexity, which is which is all about understanding the intrinsic complexity of certain tasks, and for the traveling salesman problem it belongs to a class of computational problem that’s called NP-complete. Now what that means roughly…

Speaking is that we don't have any efficient way to do it. There is no more efficient way than looking through all of the candidate solutions in order to find one, um, which means that in practice, uh, there's a huge barrier with that kind of problem if we want to try and solve them.

But then it began to appear that actually a whole bunch of problems everywhere we looked in AI, we found, okay, this problem's actually NP complete. Problems in computer vision are NP complete, endless problems in reasoning and problem solving are NP complete, or even worse. There's this big hierarchy of complexity, uh, complexity classes as they're called, where things can be even harder than NP complete problems, and then problems that are even harder than that, and and so on. And we found everywhere we looked we were encountering these problems with AI, and we hit a wall, and the wall was this barrier of uh, of combinatorial explosion. All these problems have the same character in principle; you can solve them just by looking through all of the candidate solutions to try to find the right one, one that works, but in practice it's impossible to do that.

By the mid '70s, because of the hype as well as the the series of technical problems that the AI field ran into, it went into its first, but not certain, certainly not only winter. And what that means is just funding dried up, interest dried up, people were sometimes portrayed as charlatans in the AI field, and the public just grew very suspicious of AI claims. You yourself have worked through many of these winters, boom and bus cycles. Is it almost better to work in in a winter because you get the people who are actually serious about about AI?

So, uh, for most of the time that I've been studying AI, it was a relatively quiet existence, and the nice thing about that was I just got on with my thing. There were very few people working in the same area, and as a researcher, actually that's quite a nice thing. Uh, as a researcher having, you know, a big space to yourself is actually really quite sort of refreshing. You can just explore the territory. So when the field became popular, we found huge numbers of people flooding into it. Now the nice thing about that is huge numbers of very talented people, but as a researcher what you're finding is you're no longer the only person that's looking at your problem; you're surrounded by extremely capable people all working on exactly the same problem. And so, you know, it's changed the character of doing AI research really, really quite a lot. You're not the only person at the case, you know, there's a whole bunch of people there chipping away at the same place along with you.

But you have to remember, actually, just go back two decades, and AI actually didn't have a good reputation at all. I mean, in science, AI was viewed as kind of homeopathic medicine. Neural networks were regarded as a dead field, a dead end, and I can remember colleagues saying, you know, why are you working in AI? You know, this is uh, this this is not a field that's going to be good for your career. It's just extraordinary how much that's changed.

So in the 80s, we came out of the first AI winter, and this new wave, this new paradigm of AI, was called expert systems. Tell us about how the philosophy in this second wave of AI was different from the Golden Age.

So the big idea in the second wave AI is that intelligence is primarily a problem of knowledge. And so if you want to build a machine that can do something for you, translate from French to English or to play chess or whatever, then the key problem is to figure out what knowledge the human beings use when they do that task and give that knowledge to a machine. Uh, that was the big idea: knowledge, knowledge is the key to intelligence, um, and the AI is primarily a problem of giving machines the right knowledge. And a technology emerged, rule-based systems as they were called, which made it possible to give knowledge about particular problems to machines. A classic example was the MYCIN system, which was an expert in diagnosing blood diseases in human beings. And the the way MYCIN was developed, um, was that uh, uh, the developers talked to human experts, physicians, experts in blood diseases, and they asked them, how do you go about diagnosing causes of blood diseases? And they would say, well, the first thing I do is take somebody's temperature, and then if the temperature is above this range, I would do this experiment, and so on. And that knowledge about how humans solve that problem is coded in the form of discrete chunks of what are called rules. If a human has a temperature greater than this and uh this particular blood test comes up negative and so on, then they have Lassa fever with probability 0.7. That would be an example of a rule. All of those rules were coded, given to the machine, and then you interact with the machine. The machine asks questions like, does the patient have a temperature? Have you done this test? What's the outcome of this test? And so on, and in the end it tells you, I think your patient has Lassa fever or something like that.

I see, um, but not only were new systems developed. As a a CS student myself, I was uh very surprised to to hear about a new paradigm of of uh programming that I that I haven't heard of before. Um, so when you're go to school today, you're taught two types: declarative, that's when you specify to the machine what you want, right? SQL, a lot of database languages: give me all the apples that are green and in that are from 2024. Uh, and then there's imperative languages, right? And this is Java, this is C++, this is you telling the machine, if this then do that. Uh, uh, this is what video games, for example, are made of. There's another paradigm that I just learned about in your book called logical, the logic paradigm. So let me let me give you a quote: the WARPLAN planning system, written by David Warren in 1974, which could solve planning problems, including the blocks world. So this is a simulated uh factory uh search problem we described in the Golden Age, including the blocks world and far beyond that, required just a 100 lines of Prolog code. Prolog is the logical programming language. Writing the same system in a language like Python, imperative, would be likely to require thousands of lines of code and months of effort. The temptation with logic programming is, if I just give you the fundamental truths that I know, logical deduction can elegantly take it all the way. Yeah, and that's that's a beautiful idea which beguiled an enormous number of AI researchers.

So the idea of logic programming takes symbolic AI and knowledge-based AI one step further, and it says that okay, if we want to build machines that have knowledge, the way that we give them that knowledge is by expressing that in the form of logic. We give them these these logical these logical descriptions of the world, and this is Aristotle essentially, right? If if Socrates is a man, all men are mortal, Socrates is mortal. This is Aristotelian logic 101, exactly. So we give it, but we give it all of all of the knowledge about a particular problem, whether it's diagnosing blood diseases or solving planning problems and so on, we express that in a logical form, and then inbuilt logical reasoners will sort out the details for us. They will they will do the logical reasoning. And so the idea there was in logic-based AI that intelligence is primarily a problem of deduction, of logical reasoning, and it's a beautiful and elegant idea. And the WARPLAN program, you say, with 15 lines of code, it's just it's a ridiculously short program. Problem is, it's just not very efficient, firstly, actually, and it turned out to be in many cases hopelessly inefficient for lots of problems, uh, but also it just turned out that again, you know, if you want to do robotics, Prolog is not the language you need for doing robotics; it's just completely unsuitable for that. But it would be impossible to imagine expressing all the knowledge that ChatGPT has been exposed to to manually express all that in the form of logical expressions and give that to it; it just wouldn't work.

That didn't stop one particular project you described, MYCIN, which was very limited to specifically to doctors and hospitals, and the idea was again, let's just dump all the the the first principles, the primary facts we know about the world. Logical deduction is going to figure out if Socrates is a man, then Socrates is mortal. There's another project called Cyc that attempted to store all of the knowledge that we have as a civilization into this kind of logical structure.

Yeah, so the Cyc project has a somewhat mixed place in uh, in the history of of AI. So the vision, this was the vision of Doug Lenat. Lenat was a really brilliant researcher who really dazzled people in the early 70s with uh, with his work, um, and uh, he became convinced that that the really big problem of AI, the problem of building machines which are as fully capable of human beings, is simply a problem of knowledge. And he said, there's no shortcut to this; we're just going to have to give the machine all this knowledge. So uh, he convinced some funders to support his work, and at one point they had kind of warehouses full of people busy encoding all of human knowledge in these forms of rules: if this and this and this, then this. Uh, with the idea that of eventually this would be as capable as as a human being, and and Lenat was uh was very, very optimistic about his project. He said, you know, within a couple of years, I remember reading this in the uh beginning of the 90s, he said within a couple of years Cyc is going to be smart enough that we'll it'll just be able to write its own rules, and we won't need to, we'll just give it textbooks.

Kind of like the reflexive, yeah, exactly that. Um, now the the happy part of the Cyc story is that the knowledge graphs that are used by search engines and uh behind a lot behind the scenes in a lot of search now, they trace their intellectual history to these very, very what I call very large knowledge bases. But the the Cyc was just ridiculed at the time as being just ludicrously overambitious. It never delivered anything at the scale that was anticipated for it; it found some applications, but relatively niche applications, and it never delivered anything at the scale that Lenat hoped for it. And so it was kind of often it was the ridicule, it was ridicule, but it was held up as that that one project which summarized everything that went wrong about symbolic AI. Let me give you what I think to be the funniest quote from your book: Cyc's main role in AI history is an extreme example of AI hype which very publicly failed to live up to the Grand predictions that were made for it. The founder of Cyc, Doug Lenat's role in AI has been mythologized in a piece of computing folklore: a microLenat, so the joke goes, is the scientific unit for measuring how bogus something is. Why a microLenat? Because nothing could be as bogus as a whole Lenat. Yeah, and I think this is what you were trying to get at about why it's important to study the history of AI to get the proper perspective. With every new paradigm, the Golden Age, we're so close to using search to solve everything with the expert systems, oh, like Cyc is going to be able to to basically reflect recursively write its own rules. But even when you when you go back like every technology, whether it's the printing press, people immediately wanted to write the encyclopedias, right, that captured all the knowledge in the world. This is the drive that you see in today's AI, today's AI, yeah.

Um, so I think uh, going going back to Lenat and Cyc, I mean that's kind of that that quote's kind of slightly cruel, I think. I mean, the joke is a slightly cruel quote, but the truth is, I mean, I think it would have a much happier place in AI history if it hadn't seen so many just inflated claims that were that were implausible at the time and that just weren't delivered. Uh, if it had had slightly more measured objectives, it would have held up much better as an exercise in large-scale knowledge-based development. And I say, you know, there are in the DNA of the knowledge graph behind the scenes of of Google search and so on, there was a little bit of of of Cyc there. It was all for not, it was it for nothing, but it was just these overinflated claims like, you know, it's going to start reading and writing its own rules and so on. But there is a striking, I think, analogy that you pick up on, which is the way that large language models are trained, which is that we just expose them to every bit of digital data that we can get our hands on. The difference is that in the Cyc case, human beings were interpreting all of that and writing, coding down the rules in the computer language. With large language models, none of that goes on; it is just presented to the model, and in some sense, and I'm waving my hand madly at this point, in some sense it it finds order in that, and how it does that actually we don't really understand, as we were talking about earlier. We talked about the Golden Age with search, we talked about expert systems in the late 80s. Rodney Brooks, in as a reaction almost to to the overexaggeration of of the of these expert systems, started a new paradigm called behavioral AI, uh, and some of that philosophy is behind iRobot, the company that he founded. Tell us about what the philosophy in this paradigm was.

So Brooks questioned the fundamental principles on which AI had been working since the 1950s for 30 years, and those principles were that uh intelligence uh can be solved through a process of symbolic reasoning, that we give the machine the knowledge it needs to solve a problem, and that those are the key components of intelligence. Brooks said, actually, I I just don't think that's how intelligence works in human beings, and he came up with an alternative theory, this kind of behavioral theory. And roughly speaking, what he said is we are a mass of conflicting behaviors, that some of which are genetically hardwired into us through through evolutionary processes, some of which we learn throughout our lives, but we're just a mass of these behaviors, and somehow uh human uh human intelligence arises from the interaction of those behaviors. So he said, let's start out by building layer by layer those behaviors. And he was also um extremely unhappy with the idea of intelligence being manifested in disembodied systems. He said that's not real intelligence; human intelligence is something in the world; we do things in the world. And he was deeply critical of any version of AI that wasn't capable of dealing with the world. I mean, he was a roboticist; he wanted to build robotics that could do things. So he built an architecture, a framework for doing this, where you would start with the most fundamental behaviors, the most basic behaviors imaginable. And in robotics, famously, the most fundamental behavior that you learn on day one of any robotics course is obstacle avoidance, um, because your robot crashing into things is expensive, and you'll be on the end of lawsuits and all sorts of things like that. So you you start out by building your very first layer is obstacle avoidance, and then imagine a robot that's going to go around this room picking up trash. The next level of in behavior might be exploring, right, just exploring around the room to try to find the trash, and the next level of behavior might be if you see trash you pick it up. You gradually build up layer and layer and layer. You then have to think about how those behaviors interact with one another and what takes what takes precedence. So obstacle avoidance, for example, tends to take precedence over everything. You know, if you're if it's a question of destruction versus survival, you know, you always you know, you want to choose survival. And the really cool thing is he was able to build robots that could do some quite impressive tasks in the real world as a way towards intelligence. Uh, it hit problems that were not dissimilar in spirit to the kind of problems that people had encountered when in symbolic AI and combinatorial explosion. Just we after a relatively small number of behaviors, it starts to be very hard to organize those behaviors and think about the way that those behaviors are going to interact with one another. And so it kind of reached a limit at some point by by the mid 90s, I think. But what Brooks did is he was able to build successful robotic systems. Famously, the Roomba robots uh are built using a version of, right, his ideas. And I think uh that is the best example of all the ideas that he's talking about, right? So so the the the vacuum robots essentially uh that go around in your house, they're embodied, they're robots, they're not just a software, but importantly, if you look at the programming behind the robots, it's not like a top-down search and go through the entire space; it's kind of like go straight if there's an obstacle, to take a random number, turn this amount of degrees, and map out the space. It's it's very reactive, right? And so philosophically, I think the helpful contrast between the first two paradigms, the Golden Age and the expert systems, is that those are kind of like top-down systematic, right? I'm gonna have one search, and I'm gonna search through all the combinatorial possibilities, or or I'm going to encode all of human knowledge and this one thing, and behavioral is the opposite. Yeah, it's highly reactive; it's to say, if you meet this situation, then go do this, almost like a lookup table. And at the time, people picked up on analogies with behavioral theories of psychology and Skinner, um, you know, who used to do all the experiments and training dogs and whatever by giving them stimuluses and rewards and and punishments and so on. And uh, and behavioral psychology was kind of somewhat discredited, largely discredited, I think, at the time as a as a general theory um uh of human behavior, and it people picked up on exactly those critiques and pointed exactly those critiques at at Brooks's behavioral AI, right? But that's not just, I think, what was interesting for me was it was one of the relatively few attempts to really go back to the basics of what AI is and say, how what is our fundamental guiding principle? We do this, and really there've been relatively few of those. There's symbolic AI, there's behavioral AI, there's the new AI of machine learning and deep learning and so on, which is kind of data-driven AI, right? The last paradigm in this symbolic world that I want to talk about is the early '90s agent-based AI, and my understanding here is that it's an attempted synthesis of the intuition of uh the expert systems and the Golden Age that we want our machines to be proactive, that there's a goal-directed, but in combination with the reaction uh of of the behavioral AI, in addition to a third idea of of agents being social in nature. So tell us a bit more about this agent-based AI.

So we're on home territory for me; this is what I've worked on basically my whole career. So, um, one way to think about this is changing our relationship to computer software. On Microsoft Word, everything that happens because you make it happen; you select something from a menu or click on an icon, but there's only one agent in that interaction, and it's you, and you are just telling the machine very much like, you know, you're giving detailed low-level instructions to Microsoft Word, somewhat like programming it, right? In the same kind of style. And the idea that emerged uh in the end of the 1980s, beginning of the 1990s, and which I worked on, was to change the relationship of software so that the software becomes an agent that's acting on your behalf, that's cooperating with you, working with you on the task that you set it. Um, so it's not just the dumb recipient of instructions, but it's actually now an active participant working with you uh and potentially with other agents. So to put it another way, um, the the manifestation of the agent dream that we see most obviously now is in Siri and Alexa and Cortana; they literally they they are they are direct descendants of that idea, and actually uh Siri emerged from work on agents from people that I knew working in the same community in in the 1990s. Um, so if um if we have Siri, the idea of Siri is that it is uh actively working with us on a problem rather than just being told what to do, uh, but that might involve interacting with other Series. So if I want to arrange a meeting with you, why would I call you? Why would my Siri call you? Why doesn't my Siri just talk directly to your Siri? That is the idea of what's called multi-agent systems, and that's that's kind of what's driven most of my work for the last 35 years.

I see. Well, what I found fascinating about the agent paradigm is that it almost it's agnostic to and it cuts across the symbolic modeling, the the mind and modeling the brain, because how it is proactive or how it is reactive or how it interacts with other agents, that's you abstracted away from the type of questions you're thinking about.

Yeah, so what does multi-agent systems have to offer to the current paradigm of foundational models now? Oh, wow. Well, this is we're really at the cutting edge now; we're I mean, this is a big research question is about we have large language models, and they are not sort of full general intelligence, but they are nevertheless very capable. How do we actually deploy those in our agents? Do they just handle the natural language part, the conversational part, or could we actually leverage them to do problem solving or things like that? Now we've already talked about the idea, you know, can large language models solve problems, and I'm a bit of a skeptic at the moment about the extent to which they can do that, but how exactly do we leverage this technology in the best way possible uh is is right at the cutting edge of research right now. That's exactly what people are thinking about.

So what drew you to this agent paradigm, and especially this multi-agent paradigm? Oh, well, so as an undergraduate in the 1980s, uh, I was fascinated with AI, uh, but I also became fascinated with computer networks. And you have to remember at the time, computer networks were not common. You know, the the the predecessor of the internet, the ARPANET, um, developed by the Advanced Research Projects Agency in the US, essentially military research funding agency, had, you know, a very incomplete international network with just a few nodes connected in the UK. But I got the opportunity to work on the UK's extension of that called JANET, the Joint Academic Network, and I had a kind of moment of revelation at which point I realized this is going to be the future: networks; they're just going to be everywhere; we're going to everybody is going to be using computer networks. It was obvious to me that you were going to hook up to the network through your phone or something like it, and this was going to be everywhere. And so I had those two ideas in my head: I had I'm really interested in AI; I know that networks are going to be the future; this is uh, this is obvious; it's going to be the future. Put those together and think about a network of AIs, AIs talking to one another, and that's how I got interested. Literally, that's how I got interested in in the idea, just having what happens if we got two AI systems that are capable of communicating? Well, how are they going to communicate? What's the language? What are the rules of the protocols that they're going to use? And that's what kicked off my interest. By the way, having realized that networks were the future, I completely failed to anticipate the World Wide Web or Amazon or any of that. I look I look at the missed opportunities in my life um for for doing transformational work. I totally got that networks were going to be the future, but I still didn't understand exactly what that future was going to look like.

It sounds like you were expecting us to go directly to multi-agent, like I have my AI bargaining on behalf of me, exactly, whereas first we went through this multi-almost symbolic phase where, you know, what is Amazon? If what is Google if not a big advanced search algorithm? But now, do do you think we're heading into a multi-agent world in the sense that I'm going to have my own AI agent to to act on behalf of me, that's make a lot of this uh early internet stuff obsolete?

I think it is inevitable, one way or another. I don't I think absolutely the history of computing tells us that this surely must all the lessons that we we learn from the history of computing point to the future of AI being not just one big isolated system but multiple AI systems interacting with one another, because that's how computing, the history of computing has gone. So I absolutely believe that. The problem is I don't know exactly what that's going to look like, and that's what I'm trying to figure out now; that's what my current research is trying to figure out. Um, there's another way in which multi-agent systems I imagine are are currently being deployed, even LLMs, not multiple LLMs talking to each other, but how you split work within one LLM, right? So the the rough intuition is, you know, maybe uh it's better to train actually three hidden LLMs: one's good with math, one's good with intuition, one's good with creativity or language, and then you have a a a a job sort of processing unit that gives uh the different LLMs different tasks to to process; that that also is a type of multi-agent work.

Yeah, absolutely. And those are exactly that those kind of architectural questions: how do LLMs fit? What does that architecture look like? That's again, we're right at the cutting edge; that's some people looking at those questions right now and trying to figure out what the right way to organize all that stuff is, right? And so what are some of the biggest questions uh in the field right now? What enormous numbers of people in the AI community are grappling with is is trying to get to grips with the capabilities of large language models, to really map out what these models can reliably do and what they can't reliably do, and exactly what capabilities they really do have versus those that they don't actually have. And it turns out this is really quite difficult. One of the reasons it's quite difficult is that um they've because they've essentially been exposed to all the digital content in the world, it's quite tough to come up with things that you're confident they've really fundamentally never seen before. Um, but this is uh a really exciting area of science; it's one of the one of the key areas, I think, one of the most important areas of science right now, and it goes back to this thing that, you know, AI has just become this experimental science in the way that it wasn't previously, and this is part of that picture, trying to map out these capabilities. And it's also really frustrating because these models frankly behave in slightly weird ways. You think you've got some principle or some rule one day, and then you just change your prompt slightly in ways that seem innocuous to you, and you get a completely different answer the next day, and it's uh, okay, so what went on there? What how why did it change? But mapping that out uh is is is genuinely very fascinating at the moment, right?

Um, so I want to move on to the last part of our conversation, which is I focused most of our time talking about the history on the symbolic AI side because that I I feel like it's almost a forgotten history at this point because when we think AI, we think ML and not the symbolic explicit programming side. Um, I just want to trace out and round out this history for for our viewers because what was fascinating to me was that AI people didn't use to associate AI with ML. In fact, machine learning, it seemed from your book, grew as a separate field starting in the 40s, right? This idea of can we recreate the brain structure with uh electric neurons with computation, and then the big milestones, connectionism in the 1980s, this is when we figured out backpropagation, basically a way to add more layers to to actually simulate to train these networks. Deep learning, even more layers and more scale in the 2000s, and eventually transformers, foundation models in the 2020s. What I find so poetic about this this entire history now that Mo comes full circle is what didn't work was rationally trying to explicitly tell computers what to do. What did work or what is working now, let me say, is by imitating the biological structures of the human the human brain.

Yeah, it is a remarkable uh, it's a remarkable uh change in fortunes for neural networks, which I say 20, 25 years ago was really regarded as kind of homeopathic medicine, was in some sense not taken very, very seriously, partly because of the scale that would be required to build large neural networks, and it didn't seem plausible 25 years ago that we would have computers that could process neural networks with 200 billion parameters or 500 billion parameters. And yet that's that became possible because of the computer power that we have available now, and it turns out that these these systems can be incredibly capable. So it really is a remarkable, remarkable story. I think one point in the book I say, you know, if you if you think that science is about orderly progress from ignorance to truth, absolutely is not; it's messy, false turns, um, almost kind of like ideological crusades. I mean, and it really is ideology, religious, this rounds in a full circle, the apocalyptic uh mentality of the of the ex-risk people. I want to talk about this new generation of foundation models, and I'll I'll begin with a quote from your book again: large language models, of which GPT-3 is perhaps the best known, are the most prominent example of current foundation models. While foundation models have demonstrated impressive capabilities in certain tasks, because they are inherently disembodied, they are not the end of the road in artificial intelligence.

So in the past three years, the jump from deep learning to what we have right now, foundation models, is the Transformer architecture and increasing scale, a lot and a lot of of data. Some people seem to think that the architecture is already there; we've solved it with the Transformer; we have what we need to go to AGI; all we need is more scale. What do you think is wrong about that argument?

So firstly, let me say what we've seen in the last few years in terms of Transformer architectures and that that which were released by Google, a Google lab I believe in 2017, and what they are is an architecture for token prediction and were developed in order to enable large language models so that you could give a prompt and they could predict essentially what should what should come next. So you know, the life and achievements of Winston Churchill or the history of Christ Church College, where we are now, um, and they turned out, coupled when you couple a Transformer architecture with the willingness to throw unimaginable quantities of computer power and really mind-boggling quantities of computer power to train them and mind-boggling quantities of data, you get something which was remarkable, and honestly, AI researchers that tell you that they were not surprised by how good it was, I think is is misleading you a little bit. They are genuinely remarkable; they took me by surprise; I didn't expect how good they were going to be. But just pause and think for a minute what we've got: we've got large language models that you can have a chat about quantum mechanics, the history of Christ Church College, Liverpool Football Club, uh, you know, the origins of the First World War, the economic circumstances that led to the 2008 financial crisis, or recipes for um uh for uh Arnold Bennett or whatever, uh, anything you can think of, you can you can ask these things about, and we look at that and think, wow, this is AI; this is this is intelligence. And yet we don't have a robot that could go into your house, clear the dinner table, and load up the dishwasher. Why have we got that weird dichotomy? Because there is a huge range of human activities that actually at the moment are well out of the reach of AI, and those activities are activities in the real world. Um, doing robotic AI uh is just very, very hard. Um, large language models succeed in remarkable ways, and they are genuinely impressive achievements, but they succeed on tasks where there are huge amounts of data available and in some sense where the consequences of what they do just doesn't really matter that much. You know, if you get a bad omelet recipe through ChatGPT, you get a bad omelet; that's not the end of the world. You know, you build a robot that occupies the real world with human beings, and it goes wrong, you know, it can create havoc; it can cause real harm. So an a the idea of AI which doesn't embrace doing things in the real world is quite an impoverished version of AI, I think, and that's what is uh I think interesting in your critique because you use the word disembodied, and that's kind of the the intuition, right? And that that's what you mean by disembodied. And there's something else I want to pick up on there. I mean, you're having a conversation with ChatGPT; you go on holiday for two weeks and leave it hanging; it's not wondering where you are; it's not thinking, where's Wridge got to? Or it's not getting bored or anything like that at all; it's not doing anything; it is just a computer program that's paused in a loop. Human intelligence, animal intelligence, is fundamentally different to that; we exist in a world; we're aware of the world; and that's what embodiment means; it's not just having a body, but it's actually being tightly coupled with the world we live in. In that sense, I see.

So in your Turing lectures that you recently gave, you separated out uh two general sets of human capacities: one is the embodied set, and this is uh the ability to sense one's surroundings, the ability, and there I agree with you, right? AI is just robotics lacks a lot further behind, just, you know, natural natural language processing, image processing, image generation, all sorts of stuff like that. But what I found really surprising in your Turing lectures is you also listed out a series of intellectual capabilities, and even there you didn't seem to think that our current generation foundation models are going to get us there. So the things that you said were solved or solvable in the current architecture: natural language processing, recall, common sense reasoning. But I was extremely surprised that you listed the following, again, not embodied but intellectual capacities as still not being even within the horizon uh of of current LLMs: logical reasoning, abstract reasoning, planning, arithmetic. Do you still hold that position with GPT-4? And because like it can do arithmetic, it can uh, can it can it really do arithmetic, or can it do something that looks like arithmetic?

I mean, there is a big question mark around whether um what large language models are doing is doing those things or whether they're doing something that looks like pattern recognition. So arithmetic, I'll concede you probably now is is a solved problem, but there's a huge body of work looking at whether these things can actually solve problems that are not just variations of something they've already seen in their training data. And the question of is it really originally solving a problem versus just doing pattern recognition, at the moment that's one of the big questions, and the jury is very much out on that, and the weight of evidence at the moment is they are not doing problem solving; they are doing something which is much more like pattern recognition. So let me give you an example to illustrate this. Um, so uh in AI, we've long been concerned with problem solving and planning, and planning is is the process of here is some goal I want to achieve, here is where I start out, and here are some uh choices available to you, some actions that you can perform that will transform the world. How do I organize th AC those actions to transform me from where I am to my goal? Absolutely fundamental AI capability that people have been looking at for uh for well over half a century. So can large language models do planning? First, people got very excited because it appeared that they could. You can a trip like seem exactly. But uh, on closer inspection, suppose you do the following: um, suppose you obfuscate all the terms that are being used in your in your plan, so you don't use words that it's familiar with,

That the deeper issue you're gesturing at, so that's what Transformers were designed for: next word prediction. And the surprising thing was how useful and impressive that turned out to be, if you were prepared to throw enough data and compute power at it. Um, but I see no reason to believe that the Transformer architecture is the key, for example, to robotic AI. That's not what it was designed for, so I don't see why it should—Lal reasoning or logical reasoning—necessarily. I mean, again, that's not what it was designed for, but—iiz—that doesn't mean it's not useful. And I'm as dazzled as anybody when I use—when I use—this technology, and I am, you know, daily taken by surprise when people show me the really remarkable things that it can do.

Um, and I have to say, you know, we've gone—this is really genuinely, I think, a watershed moment in AI history—because we've gone from a period where a lot of questions in AI were purely philosophical questions; they were literally reserved for philosophers until a few years ago. Uh, and suddenly it's experimental science. You know, are large language models conscious? Well, let's roll up our sleeves and do some experiments and find out—no, by the way, they're not. Uh, but you know, these are now practical, hands-on questions. And to have gone from not having anything in the world that you could apply those questions to, to this being actual practical, hands-on experimental science in just a few years is mind-blowing.

Right, let me play a devil's advocate here. Um, because for me, as a philosopher, studying AI has been a very humbling experience, because it might reveal what—how little—reason actually works in humans. And here's the challenge I would like for you to respond to, which is: these architectures are built off of an imitation of the human mind, and how the human mind is—is connected—through neural networks, right? And so the intuition is: maybe by—by just imitating that—even though they're not designed for logical reasoning—because we've imitated the structure of the human brain—that it's this emergent phenomenon. Know, maybe what humans are doing is not first principles thinking; maybe we're—we're just pattern—pattern matching. Maybe it's all pattern matching down there. Maybe—I mean, I—I believe—I don't really believe this one—but maybe we are just doing next word production when we're having a conversation. And there is an entirely serious school of thought that thinks actually perhaps we need to rethink what the—the—what humans are doing, and that actually that we have overblown expectations about what beliefs—about what we're—what we're doing.

I don't think humans are a Transformer architecture. I don't think that's what we're doing. I think there's a lot—lot more that's going on. We are animals that have evolved to inhabit planet Earth and to interact with other human beings and to understand the fundamentals of human nature. I think you have to understand those two things. Transformer architectures are not that—not by a long, long, long way. Um, however, the point you make about emergence I think is an entirely valid one. We don't understand how intelligence emerges in human beings. How does all that gooey stuff in our heads—all those electrochemical processes and so on—give rise to you and me? We don't understand that in a deep way at all. And that's what's so exciting about the present time—that let's roll up our sleeves and find out how it's actually doing this.

But without wishing to denigrate these systems at all, there is a very real sense in which they are a hack. They are an engineering hack that's put together. They are not following some deep model of mind or some deep philosophical theory about how human intelligence is—or—or some deep cognitive science theory of human intelligence. Um, they are a technological hack. Um, and although neural networks—artificial neural networks—were inspired by the structures we see in human and animal brains, they are not an attempt to faithfully recreate that in people. There have actually been attempts to do that; there was a very large European-funded project that wanted to try to recreate a brain—an actual brain—but that's not what neural networks are doing.

Given how much success we've had about imitating a specific structure of brain—right—how—how neurons are—are linked together in computation—should we be looking more into biomimicry, and should we be studying the brain more and see if there's other structures we can replicate? Is that the path forward to finding out the architecture is to take us to—to? I think that's one—that's one way forward, and I think we will surely get some insights. I mean, we have a very incomplete understanding of how the brain is organized. I mean, the brain is not just one big homogeneous neural network, even though, you know, it contains vast neuro—multiple neural networks, but it has—it has some functional structure, and we—we understand a lot more now than we did even 30 years ago about the functional structure of the brain, but a very incomplete understanding. So that's going to be one way to go, but I emphasize again, you know, we are great apes that have emerged through a process of billions of years of evolution to inhabit planet Earth at ground level—at sea level—roughly speaking—and to be able to—to learn about the physics of planet Earth and to be able to operate within the dynamics of the physics of planet Earth, but also to be able to interact with other great apes—and those are the two key big components of human intelligence: learning about our world and learning about other apes—human beings. Um, and we shouldn't lose sight of that when we think about the successes that AI has had. You know, we—we're not just a big neural network—a big homogenous neural network—there's an awful lot more going on than that, right.

Um, but there is, I think, something melancholy about what AI techniques have worked and what haven't. And let me quote to you a lovely quote from your book—this is you speaking in your voice—in July 2000, I was at a conference in Boston watching a presentation by one of the bright young stars of the new AI. I think this is when ML was starting to—to pick up steam. I was sitting next to a seasoned AI veteran—someone who had been in AI since the Golden Age—a contemporary of McCarthy and Minsky. He was contemptuous—"Is this what passes for AI nowadays?" he asked, "Where did the magic go?" You speaking. And I could see where he was coming from. A career in AI now demanded a background not in philosophy or cognitive science or logic, but in probability, statistics, and economics. This, I think, is what you were getting at about not seeming as poetic. You know, you—you thought with—with psych and artian logic—that's what was going to do it. Well, it turns out like the neural net architecture—with a black box that we can barely understand—more than our brains—a lot of mundane mathematics. Exactly. And you think we need more clever architecture to get our neural nets to behave differently? Well, it's—it turns out just increasing the scale fundamentally changes the—the increasing output of—of the behavior. Oh, that is a—that's a very depressing lesson. I mean, the fact that, you know, you would think we—the chief source of advances in AI is scientific developments—actually, no, it's just more compute, more data. Um, and there's—there's—there's an article by this—called Rich—by a guy called Rich Sutton called "The Bitter Lesson," and Rich is a very renowned machine learning researcher, and he said, "Look, the truth is we've made progress primarily in AI by—you know, some core ideas, but actually the—the big steps in progress we've seen—and when we've been willing to throw 10 times more compute—10 times more data—and so that—that is a sobering lesson.

But there is still magic there in AI. I mean, so—the fact now that we have machines like ChatGPT that we can have a conversation with—that we can turn the conversation to anything that we might care to imagine—compared to where we were five years ago—that is simply astonishing. And you know, uh, if I wish I was a PhD student now and having the opportunity to explore this kind of weird new landscape of AI and to try to figure out, you know, what are the—what are the fundamental laws that govern these systems—what are the principles—try to uncover the science underneath this—this technology. Um, there is still some magic there; you just have to look a bit harder to find it. It might be a bit better if we needed the most advanced math, or we need to invent this fancy architecture, to study human brains very closely for decades. Um, but I think I love your word "sobering," because that's another way to frame melancholy or disappointing. And I'll end on this—this one observation, which is: I'm preparing a lecture on the Stoics right now, and the Stoics famously think that humans are extremely rational creatures—that even unbeknownst to us, when I desire something, I'm making an implicit proposition that that thing is good. Behind most human behaviors there's an explicit—or sorry—there's an implicit true or false proposition. Someone on the opposite extreme is probably someone like Freud, where our unconscious is not known and perhaps even greatly unknowable to us. And I think the fact that neural nets—these black boxes that we ourselves don't really understand—have gotten so much more success than something explicit like psych also tells us perhaps a sobering lesson about how our own intelligence works. And that, to me, again coming from a philosophical perspective, is the most exciting stuff about all of this, which I'll go back to—which is that what were once philosophical questions have now become experimental science. All right, thank you, Professor. Thank you for a fascinating interview. Thanks for watching my interview. If you like this conversation, I think you'd also enjoy my discussion with Nick Bostrom on his new book, *Deep Utopia*. It tries to imagine what there is left for humans to do after AI has surpassed us in all domains. Now these interviews are a part of an AI series that I'm producing as a fellow of the Cosmos Institute, a nonprofit studying philosophy and artificial intelligence. You can find links to our website, the Bostrom interview, and everything else we cover today in the description below. Thank you.