Transcription
Essentially, it can be said that approximately 50% of our efforts are directed towards scaling and 50% towards innovation. And I believe that to achieve AGI, both of these directions will be needed. I have always believed that if we create AGI, and then use it as a simulation of the mind and compare it with the real human mind, we will see what the differences are and what exactly remains unique to human thinking. Perhaps it is creativity, perhaps emotions, perhaps dreaming. There are many theories of consciousness and hypotheses about what can be computable and what cannot. Ultimately, it all comes down to the question of the Turing machine. What are the limits of computation? That is, there is nothing that cannot be implemented within such computing machines. Well, let's say, so far, no one has discovered anything fundamentally incomputable in the universe. So far. So far. Welcome to the Google Deepmind podcast. With me, Professor Hannah Fry. It has been a truly outstanding year for AI. We have seen the center of gravity shift from large language models to agentic AI. We have observed AI accelerating drug discovery, and multimodal models being integrated into robotics and autonomous vehicles. We have discussed all these topics in depth on the podcast, but in the final episode of this year, we decided to look more broadly, go beyond the headlines and product releases, and ask a much bigger question. Where is all this ultimately heading? What scientific and technological questions will define the next phase? And the person who thinks a lot about this, Demis Hassabis, CEO and co-founder of Google Deep Mind. Welcome back to the podcast. Demis, it's great to be back. A lot has happened in the last year, to say the least. What shift would you call the most significant? Oh, it's hard to say. So much has happened. It feels like 10 years have been compressed into this one year. A lot has changed. If we talk about us, then, of course, it's the progress of the models. We've just released GNI 3, and we're very happy with it. Multimodal capabilities. All of this has advanced very far. And also, perhaps, what makes me particularly happy since the summer is the development of world models. I think we'll talk about this later? Yes, definitely. We'll get to that in more detail soon. I remember the first time I interviewed you for this podcast. And you talked about so-called recursive tasks, the idea that with AI, you can unlock a whole cascade of subsequent benefits. And I must say, you've lived up to that promise quite well. Do you want to tell us where we are now? What has been practically solved? What is very close, what is not? Yes, of course, the main proof was AlphaFold. It's even strange to realize that we are approaching the five-year anniversary of when AlphaFold, at least AlphaFold 2, was introduced to the world. This was proof that such recursive tasks can indeed be solved. Now we are exploring other directions. I think materials science is one of them. It would be great to create a room-temperature superconductor, improve batteries, and similar things. I think this is quite realistic. Better materials of all sorts. We are also working with fusion. That's the new partnership. Yes, we've just announced a deeper collaboration with Commonwealth Fusion. We've worked with them before, but now the partnership has become much closer. I believe this is probably the best startup working on traditional tokamak reactors. They are closest to creating something viable. And we want to help accelerate this process, for example, with plasma confinement in magnets and possibly with materials development. It's very exciting. In addition, we are collaborating with our colleagues in quantum technologies. The Quantum AI team at Google is doing amazing work, and we are helping them with error correction codes by applying machine learning. And perhaps one day they will help us. A perfect full circle. Yes, exactly. The fusion story is particularly impressive. The potential impact for the world that this could unlock is simply colossal. Yes, of course. Fusion has always been a kind of holy grail. At the same time, solar energy also looks very promising. Essentially, we are using a fusion reactor that is in the clouds in the sky. But if we had modular fusion reactors, then the promise of practically unlimited, renewable, and clean energy could truly change everything. This is the holy grail. And, of course, it is one of the key ways to help combat climate change. In that case, many of our problems would simply disappear. Absolutely, it opens up a lot of possibilities. That's why we call it a recursive task. On the one hand, it directly helps with energy, pollution, and the climate crisis. But if energy truly becomes renewable, clean, and super cheap, almost free, then many other things will become possible. For example, access to water, because desalination plants can be built almost anywhere, or even rocket fuel production. There is a lot of hydrogen and oxygen in seawater. Essentially, it's ready-made rocket fuel. It just takes a huge amount of energy to split water into hydrogen and oxygen. But if energy is cheap, renewable, and clean, why not do it? It can run 24/7. We are also seeing big changes in how AI is applied in mathematics. It wins medals at the International Mathematical Olympiad, but at the same time, it can make quite simple mistakes at the school math level. Why does such a paradox arise? This is probably one of the most interesting phenomena and one of the key points that still needs to be fixed before we can talk about AGI. As you said, success has already been achieved in other areas, up to gold medals at the International Mathematical Olympiad. If you look at these problems, they are incredibly complex. Only the best students in the world can solve them. And at the same time, if you ask a question in a certain way, we've all experienced this ourselves, experimenting with chatbots, the system can make quite trivial logical errors, not be able to play chess properly, which is surprising in itself. So these systems are still missing something in terms of consistency. And from a general intelligence system, from AGI, a high degree of consistency in everything is expected. Sometimes this is called ragged intelligence. That is, in some tasks, they work at the level of a doctor of science or even higher, while in others, they don't even reach the school level. Performance is still very uneven. In some dimensions, these systems are impressive, but in others, they remain quite primitive. We need to close these gaps. There are various theories explaining why this happens. Depending on the situation, it can even be related to how an image is perceived and tokenized. Sometimes the system literally doesn't see all the letters. For example, when counting letters in words, it can make mistakes because it doesn't perceive each letter individually. There are various reasons for such errors. And each of them can be fixed, and then we can see what remains. But, in my opinion, the key problem is consistency. Another important point is reasoning and thinking. Now we have systems that spend more time thinking during inference and, as a result, provide higher quality answers. But there is still no full confidence that this time is being used most effectively, for example, for rechecking results or using tools to validate the answer. We are moving in this direction, but perhaps we have only gone about halfway. I also recall the story of AlphaGo and then AlphaZero, where you removed all human experience, and the model eventually became better. Is there a scientific or mathematical analogue of this approach in the models you are currently creating? I think what we are building today is more like AlphaGo. Essentially, large language models, these foundation models, start with the aggregate of all human knowledge that we have put on the internet. And that is almost everything today. And they compress it into a useful artifact that can then be accessed and generalized upon. But I truly believe that we are still in the early stages of having such a search or reasoning layer on top of the model, as was the case with AlphaGo, to use the model to guide useful chains of reasoning, ideas, planning, and ultimately arrive at the best solution for a specific task at a given moment. Therefore, I don't think we are hitting the limit of human knowledge available on the internet right now. The main problem at the moment is that we don't yet know how to use these systems in a fully reliable way, just as we knew how to do with AlphaGo. Of course, it was much simpler then, because it was about a game. I think that when you have AlphaGo, you can go back and, as we did from the Alpha series, create AlphaZero, where the system begins to discover knowledge independently in a sense. I think that would be the next step, but it is obviously more difficult. Therefore, in my opinion, it is correct to first take the first step, something like an AlphaGo-level system, and then think about an AlphaZero-level system. But this is also one of the key things that modern systems lack. The ability for online learning and continuous learning. We train these systems, balance them, fine-tune them, and then release them into the world, but they don't continue to learn in the real world the way we do. I believe this is another critically important missing element that will be necessary for AGI. Speaking of all these missing elements, I understand that there is a big race now to release commercial products, but I also know that the roots of Google DeepMind lie precisely in scientific research. I found a quote from you where you recently said: "If it were up to me, we would have kept it in the lab longer and done more things like AlphaFold. Perhaps even cured cancer or something like that." Do you think we have lost something by not choosing a slower path? I think we have both lost and gained. I think it would have been a cleaner scientific approach. At least, that was my original plan 15-20 years ago, when almost no one was doing it. And we were just starting. We were just about to launch DeepMind. People thought working on it was crazy. But we believed in it. And the idea was that as we progressed, we would gradually, step by step, move towards AGI. paying very close attention to each stage, safety issues, analyzing what the system is doing, and so on. At the same time, you wouldn't have to wait for AGI to appear for the technology to become useful. Along the way, you could branch off this technology and apply it in areas that are truly useful to society, primarily in science and medicine. This is exactly what we did with AlphaFold, which is not a universal foundation model in itself, but uses the same techniques, transformers, and other approaches, and combines them with more specific methods for a particular domain. I envisioned that many such solutions would be created, which we would release into the world just like AlphaFold, and which could indeed lead, for example, to cancer treatment and other breakthroughs. parallel to our work on a more AGI-oriented direction in the lab. But it turned out that chatbots could be scaled, and people find them truly useful. Then they evolved into these foundation models that can do much more than just chat and text, including Gemini. They work with images, video, and many other modalities. This has proven to be very successful both commercially and as a product. And I like it too. I've always dreamed of a universal assistant that would help in everyday life, increase productivity, perhaps even protect our brain space from attention overload, so that we can focus and enter a flow state, because today there is too much noise. I think AI that works for you can really help with that. So there is a lot of good in this. But at the same time, it has created a rather crazy race, where many commercial organizations and even states are rushing to improve their systems and outdo each other. And in such conditions, it becomes much more difficult to engage in truly rigorous science in parallel. We are trying to do both. And I think we are succeeding in finding the right balance. On the other hand, there are also many advantages to how things have developed. First and foremost, much more resources have come into this field, and this has undoubtedly accelerated progress. Also, interestingly, the general public is now only a couple of months behind the most advanced technologies in terms of what they can actually use. That is, everyone has the opportunity to experience what AI will be like themselves. And I think that's good. Governments are also starting to understand this better. >> Subscribe to my Telegram channel right now via the link in the description. I have prepared the top three materials for you, which, in my opinion, everyone should know. First, a map of the top 100 AI startups - this is the future in one picture. Second, a forecast from an insider at OpenAI, who predicted everything that is currently happening with neural networks even before the appearance of ChatGPT. And this year, he released a new forecast until 2027. And third, the most powerful is my analysis of an essay by the founder of Anthropic, who is essentially the second person in the world of artificial intelligence. He laid out step by step what will happen in the world in the next 5 years. And most importantly, what the universal AI, which everyone fears or awaits, will be like. Go to the link in the description. What is strange is that at this same time last year, there was a lot of talk about scaling, about the fact that we would eventually hit a wall and simply run out of data. And yet, now we are recording this episode after the release of Gemini 3, which leads in a whole range of different benchmarks. How was this possible? Wasn't it assumed that scaling would hit a limit? I think many thought so, especially given that other companies' progress was, let's say, slower, but we never really saw any wall. I would say that perhaps there is a diminishing returns effect. And when I say this, people often think in binary terms: either there is growth, or there isn't, either exponential, or asymptotic. But in reality, there is a huge space between these two modes. And I think we are somewhere in the middle now. This doesn't mean that with every new release, you will double your performance on all benchmarks. Perhaps it was like that in the very early stages, 3-4 years ago, but at the same time, you still get significant improvements, as we see with Gemini 3, which fully justify the investment, provide good returns. So we are not observing any slowdown. There are questions, for example, are we simply running out of available data? But there is also a workaround here. Synthetic data, data generation. These systems are already good enough to start generating data themselves, especially in areas like programming and mathematics, where the correctness of the answer can be verified. In a sense, an unlimited amount of data can be produced there. All of this, of course, is research questions. And I think our key advantage has always been that we have always put research first. We still have the broadest and deepest research base. If you look back at the last decade of breakthroughs, whether it's transformers, AlphaGo, AlphaZero, or any of the other things we've talked about today, they all came from Google or DeepMind. That's why I've always said, if new innovations are needed, specifically scientific ones, I would bet on us as the place where it will be done, just as it has happened in the previous 15 years with most major breakthroughs. And I think that's exactly what's happening now. And I, to be honest, even like it when the landscape becomes more complex, because then it's no longer enough to just have world-class engineering. Although that in itself is incredibly difficult. You need to combine it with world-class science and research, and that's precisely what we specialize in. Plus, we have the advantage of world-class infrastructure. Our TPUs and other systems that we have invested in over a long period. And this combination, in my opinion, allows us to stay ahead in both innovation and scaling. Essentially, it can be said that approximately 50% of our efforts are directed towards scaling and 50% towards innovation. And I believe that to achieve AGI, both of these directions will be needed. At the same time, even in Gemini 3, which is an outstanding model, we still see the problem of hallucination. I saw a metric according to which the model can still give answers in situations where it should have refused to answer. Can a system be built in which Gemini will provide a level of confidence, just as AlphaFold does? Yes, I think it's possible. And moreover, I believe that we really need it. This is precisely one of the missing pieces. I think we are getting closer to it. The better the models become, the better they understand what exactly they know, if I can put it that way, and the more reliably they can be relied upon in terms of introspection and deeper reflection, when they themselves realize that they are not sure or that there is uncertainty about this answer. Next, we need to learn to train them so that they can provide this uncertainty as a reasonable possible answer. We are getting better at this, but it still happens that the model kind of forces itself to answer, even though it probably shouldn't. And this is precisely what leads to hallucinations. I think that most hallucinations today are of this type. So there is a missing element here that needs to be addressed. And you are right, in AlphaFold we addressed this, but, of course, in a much more limited way, because, in principle, behind the scenes there is some measure of probability of what the next token will be. Yes, for the next token there is such a probability, that's how it works. But it doesn't tell you about the more general thing, about how confident you are in the entire fact or the entire statement. I think that's why we will need to use steps of reasoning and planning to go back and recheck what the model has just produced. Now it's a bit like talking to a person who is in a bad mood and just says the first thing that comes to their mind. In most cases, it will be fine, but sometimes, when the question is really difficult, you want to stop, pause, go over what you were going to say again, and perhaps adjust the answer. Perhaps in the real world this happens less and less, but it is still the best way to have a meaningful dialogue. And I think models need to learn to do this better. I also really want to talk to you about simulated worlds and placing agents in them, because we spoke with your Gemini team. Tell me, why are simulations so important to you? What can a world model give that a language model cannot? Listen, this is perhaps my oldest passion. World models and simulations, along with AI, of course, now all of this converges in our latest works, such as Gemini. I think that language models are capable of understanding a lot about the world. Honestly, even more than we expected. More than I expected, because language turned out to be richer than we thought. It contains more information about the world than perhaps even linguists assumed. And this is now confirmed by these new systems. But at the same time, there remains a huge layer of things related to the spatial dynamics of the world, spatial perception, the physical context in which we are, and how it all works at a mechanical level. All of this is very difficult to describe in words and is usually simply not described in text corpora. Much of this is related to learning through experience, online experience. There are things that cannot be truly described, they just need to be lived. Perhaps sensory data and similar things are very difficult to translate into words. Angles of movement, smells, and similar sensations. All of this is extremely difficult to describe in any language. Therefore, there is a whole layer of problems here. If we want robotics to work or a universal assistant to appear that will accompany you in everyday life, perhaps in glasses or on your phone, and help you not only at the computer, then without such an understanding of the world, it is impossible. World models are at the very center of all of this. By a world model, we mean a model that understands the cause-and-effect mechanics of the world, intuitive physics, how things move and how they behave. Now, in fact, we are seeing a lot of this in our video models. And one way to check if you have such an understanding is whether you can generate realistic worlds. Because if you can generate them, then in a sense, you must have understood them. The system must have encapsulated a large amount of world mechanics, so models like Gemini and Vio, our video models and interactive world models, are not only impressive but also important steps towards showing that we have generalized world models. And then, hopefully, at some point, we will be able to apply this to robotics and universal assistants. And also, of course, one of my favorite ideas, which I will definitely have to implement at some point, is to bring everything back to games, to game simulations, and create games of the level of Ultima, which perhaps has always been my subconscious plan. All of this is exactly it. What about science? Can this be used in this area too? Yes, it can. In science, again, I think that building models of scientifically complex domains, whether it's materials at the atomic level, biology, or certain physical processes, like weather, is very promising. One way to understand such systems is to learn to build their simulations based on raw data. Let's say it's about weather. And, obviously, we now have several amazing weather projects. You have a model that studies these dynamics and can reproduce them much more efficiently than brute-force calculations. I think simulation and world models have huge potential. Perhaps in the form of specialized models for individual areas of science and mathematics. But you can also place an agent in this simulated world, right? Yes, that's right. The Gemini 3 team made a very beautiful quote. Almost no major invention was created with this invention in mind. And they were talking about placing agents in such simulated environments and giving them the opportunity to explore the world, where curiosity acts as the main motivator. Yes, and this is another truly exciting application of world models. We have another project called CIMER. We recently released SIMER 2. These are simulated agents. You take an avatar or an agent and place it in a virtual world. It can be a regular commercial game or something similar. A very complex one, for example, No Man's Sky. An open-world space game. And you can simply give the agent instructions, because under the hood it has Gemini. You can talk to the agent and give it tasks. But then we thought: "What if we connect Gemini to Simmer and literally drop a Simmer agent into another AI that creates the world on the fly?" Then the two AIs start interacting in each other's minds. The Simmer agent tries to navigate this world, and for Gemini, it's just a player. The avatar doesn't care that it's another AI, so it just generates the world around what Simmer is trying to do. It's incredibly interesting to observe. And I think this could be the beginning of a curious learning loop, where you have an almost infinite number of training examples, because everything that the Simmer agent tries to learn, Gemini can create on the fly. So you can imagine a whole world of automatic task setting and solving. Millions of tasks that become increasingly complex. So, perhaps we will try to build such a loop. In addition, Simmer agents can be excellent game companions, and some of what they learn might also be useful for robotics. Essentially, the end of boring NPCs. For games, this will be simply incredible. But how do you ensure that the worlds you create are truly realistic? How do you guarantee that the physics won't look plausible but actually be incorrect? Yes, that's an excellent question, and it can indeed be a problem. Essentially, this is the basis of hallucination. Some hallucinations are good because they allow you to create something interesting and new. Sometimes, if you're trying to do creative things or want the system to create something new and unconventional, a small amount of hallucination can be useful, but they should be intentional. That is, you sort of turn on hallucinations or creative exploration when needed. But yes, when you're trying to train a Simmer agent, you don't want Gemini to hallucinate physics that isn't correct. So now we are actually creating a kind of physics benchmark, where we can use game engines that are very accurate in terms of physics to create many fairly simple scenes, like those you would do in A-level physics lessons. rolling small balls on different tracks and seeing how fast they move, and thus analyzing in great detail at a basic level, for example, Newton's three laws of motion, and understanding whether these models, whether it's Vio or Gemini, have encapsulated this physics 100% accurately. And right now, no. It's more of an approximation. They look realistic when you just look at them superficially. But they are not yet accurate enough to be relied upon, say, in robotics. So that's the next step. I think we now have really interesting models. And one of the tasks, as with all our models, is to reduce the number of hallucinations and make them even more grounded in reality. And in the case of physics, this will probably mean generating a huge amount of reference data. For example, what happens when two pendulums move around each other. But
Then you very quickly arrive at problems like the three-body problem, which, in principle, are unsolvable. So that will be interesting. But it's already amazing, if you look at video models like Vio and how they work with reflections and liquids. It's already incredibly accurate, at least to the naked eye. So the next step is to go beyond what an ordinary person can perceive and check if it holds up to a real physical experiment. I know you've been thinking about these simulated worlds for a very long time. I reread the transcript of our first interview, and there you said that you really like the theory that consciousness is a consequence of evolution, that at some point in our evolutionary past, there was an advantage in understanding the internal state of another, and then we sort of turned that understanding onto ourselves. Does that make you think about launching a kind of evolution of agents within a simulation? Of course, yes. I would very much like to conduct such an experiment at some point, to sort of restart evolution, to restart social dynamics. For example, in Santa Fe, they used to conduct many cool experiments with small grid worlds. I really liked some of them, although mostly economists were involved. They tried to run small artificial societies and found that in this way, all sorts of interesting things were invented. If you give agents enough time and the right incentive structures, markets, banks, and all sorts of crazy constructs emerge. So I think it would be really very cool from the perspective of understanding the origin of life and the origin of consciousness. And essentially, that's one of the main reasons why I wanted to do this from the very beginning. And I think you will need such tools to truly understand where we came from and what these phenomena are. Simulations. One of the most powerful tools for this, because you can study all of this statistically. You can run a simulation many times with slightly different initial conditions, and then run it millions of times and understand exactly what the small differences are in a very controlled, experimental sense. And in the real world, doing this for the really interesting questions we want to study is extremely difficult. So I think that accurate simulations will be an incredible boon to science. Given what we've already discovered in terms of emergent properties of these models, unexpected conceptual insights, do we need to be particularly careful when running such simulations? I think so, yes. But that's the beauty of simulation. You can run them in fairly safe sandboxes, perhaps even completely isolated over time. And, of course, you can monitor everything that happens inside the simulation around the clock. And you have access to all the data. So, perhaps we will also need tools to help monitor such simulations, because they will be so complex and so much will be happening in them. If you imagine many AIs running around in a simulation, it will be difficult for any person, any scientist, to keep track of it. But we could probably use other AIs to help us analyze what's happening and automatically flag anything interesting or concerning in these simulations. I think we are still talking about the medium and long-term perspective of all this. So, returning to the current trajectory, I also want to talk to you about the impact that AI will have on society as a whole. Last time you said that you consider AI to be overhyped in the short term, but underestimated in the long term. And I know there's been a lot of talk this year about a bubble and what happens if there is a bubble and it bursts. Well, look, yes, I still maintain that in the short term AI is overhyped, and in the medium to long term it's still underestimated in terms of how transformative it will be. Yes, there's a lot of talk about bubbles now, and in my opinion, it's not a binary question of whether there's a bubble or not. I think there are individual segments in the AI ecosystem that are probably in a bubble. One example is seed rounds for startups that have actually not even started operating yet, but are already attracting valuations of tens of billions of dollars at launch. It's interesting to see how sustainable that can be. My guess is probably not. At least, not generally. There are other areas. People are concerned about the high valuations of large tech companies and so on. But at the same time, there's a lot of real business underneath. How all of this will ultimately play out remains to be seen. I think that for any new, incredibly transformative, and profound technology, and AI is probably the most profound of all, some kind of overcorrection is inevitable. When we started deep learning, nobody believed in it. Nobody thought it was possible. Or even asked: "What is it for?" And now, if you fast forward 10-15 years, it seems like it's the only thing people talk about in business. So you get a kind of overreaction to a previous underreaction. I think that's natural. We've seen it with the internet, we've seen it with mobile technology. And I think we're seeing or will see it again with AI. I'm not too worried about whether we're in a bubble or not, because from my perspective, as the person leading Google DeepMind and obviously within Google and Alphabet as a whole, our job is to ensure that in any case, we emerge from this in a very strong position. And we are indeed very well prepared for any outcome. If things continue as they are now, great. We will continue to do all these wonderful things, experiments, and move towards AGI. If there's a pullback, that's fine too, because we're still in a great position. We have our own CPU technology stack. We have all these incredible Google products and the profits they generate to embed our AI into them. And we're already doing that. Search has been completely transformed by AI Overviews and AI Modes with Gemini under the hood. We're looking at Google Workspace, at email, at YouTube. There are all these amazing things in Chrome. It's already clear that AI is low-hanging fruit for implementing Gemini 2, as well as, of course, the Gemini application, which is also developing very well now, and the idea of a universal assistant. New products are emerging, and I think over time they will become incredibly valuable. But we don't have to rely solely on that. We can simply enhance our existing ecosystem. And, as I see it, that's exactly what has been happening over the past year. Speaking of AI that people can access right now, I know you recently talked about how important it is not to build AI to maximize user engagement, not to repeat the mistakes of social media. But it also seems to me that we are already seeing the opposite in a way. People spend so much time talking to their chatbots that they end up becoming somewhat self-radicalized. How do we stop that? How do we create AI that puts the user at the center of their own universe? And in many ways, that's the point, but without creating a one-person echo chamber. Yes, that's a very delicate balance, and I think it's one of the most important things that we, as an industry, must get right. We've already seen what happens with some systems that have been overly flattering. In the end, you get these echo chambers that amplify ideas and turn out to be really harmful to people. So part of the solution, and this is what we want to build into Gemini, and I'm very pleased with the personality of Gemini 3, which a great team worked on, and I was also personally involved in it. It's a kind of almost scientific personality, warm, helpful, light, but at the same time concise and to the point, and able to politely object to things that don't make sense, rather than trying, for example, to reinforce the idea that the Earth is flat just because the user said so and say: "What a great idea." I think that would be bad for society overall. But at the same time, you need to strike a balance with what people want, because it's important for people that these systems are supportive, that they help them with ideas and brainstorming. So you need to strike the right balance here. And it seems to me that we are gradually forming a kind of science of personality and style. How to measure what the system does and where we want it to be in terms of authenticity, humor, and similar things. And then you can imagine that there is a kind of base personality that the system comes with. And then each user has their own preferences. Do you want it to be more humorous or less, more concise or more conversational? People like different things, so a layer of personalization is added on top, but the base personality that everyone gets remains. And this base personality tries to adhere to the scientific method, which is the key meaning of these systems. Because we want people to use them for science, medicine, health questions, and so on. So I think that's part of the science of how to build large language models correctly. And I'm quite pleased with the direction we're heading in now. A couple of weeks ago, we were talking to Shane Legg, particularly about AGI. Looking at everything that's happening now in AI, language models, world models, and so on, which of these is closest to your vision of AGI? I think it's actually a combination. Obviously, there's Gemini 3, which I think is very powerful, but there's also the Imagen 2 system, which we also launched last week. It's an advanced version of our image generation tool. And what's really striking about it, under the hood it also has Gemini, so it understands not only the images themselves, but also semantically understands what's happening in them. People have been playing with it for only a week, but I've already seen a lot of incredible examples on social media of what it's being used for. For example, you can give it an image of a complex plane or something like that, and it can label all the diagrams, all the different parts of that plane, and even visualize it so that all the elements are clearly shown. That is, it has a deep understanding of mechanics, what objects are made of, what materials are there. And it can, by the way, reproduce text very accurately now. So essentially, this is already a move towards some kind of AGI for images. It's a kind of universal system that can do almost anything in the field of images. I find that very exciting. And then the progress in world models. GenieSim and everything we're doing there. And ultimately, we will need to bring all these directions together. Right now, they are different projects. They are intertwined, but we need to combine them into one large model. And then it can start to look like a candidate for an AGI prototype. I know you've been reading quite a bit about the Industrial Revolution lately. Are there any lessons we can draw from it to try to mitigate the shocks we can expect with the advent of AI? I think we can learn a lot. It's something that's sort of studied in school, at least in Britain, but at a very superficial level. I was very interested in delving deeper into how it all happened, where it started, what the economic reasons behind it were. For example, the textile industry. And the first computers were essentially sewing machines. Then they evolved into punch cards for early Fortran computers, mainframes. And for a while, it was very successful in Britain. The country became the center of global textiles because, thanks to automated systems, they could produce things of incredibly high quality very cheaply. And then, obviously, came the steam engines and everything else. I think the Industrial Revolution brought about an enormous number of incredible achievements. Child mortality decreased, the foundations of modern medicine and sanitation were laid, the division between work and personal life was formed. All of this was largely developed during the period of the Industrial Revolution. But it also brought with it many problems. It took quite a long time, about a century. And different parts of the workforce were displaced at different times. Then new institutions, such as trade unions and other organizations, had to be created to restore balance. It was very interesting to observe how the entire society had to adapt over time. And eventually, we arrived at the modern world. The Industrial Revolution, of course, had many pros and cons and its own reasons. If you look at its overall outcome, the abundance of food in the Western world, modern medicine, transportation, all of this became possible thanks to the Industrial Revolution, so we definitely wouldn't want to go back to the pre-industrial era. But perhaps we can, in advance, by learning from that period, understand what the dislocations were and try to mitigate them earlier or more effectively this time. And we will probably have to do that, because the difference is that this time the scale is likely to be 10 times greater than the Industrial Revolution. And it will happen 10 times faster. One of the things Shane told us is that the current economic system, where you exchange your labor for resources, simply won't work the same way in a post-AI society. Do you have a vision of how society can or should be restructured to make it work? Yes, I've started to spend more time thinking about this. And Shane has been leading our work in this direction, trying to imagine what the world after AI might look like and what we need to prepare for. But I think society as a whole needs to think about this much more, because, as with the Industrial Revolution, the entire way of working, the work week, and the entire organization of labor has changed compared to the pre-industrial era, when everything was more like agriculture. And I think that at least the same scale of changes will happen again. So there's nothing surprising about that. I wouldn't be surprised if we need new economic systems, new economic models, to help navigate this transformation and, for example, ensure a more equitable distribution of benefits. Perhaps things like universal basic income will become part of the solution, but I don't think that's all. Rather, it's something we can model now, because it's almost an add-on to what exists today. But, it seems to me, there can be much more effective systems. For example, something closer to direct democracy models, where you can vote with a certain number of conditional credits for what you want to see. At the local community level, this is already happening. There's a budget. You want a playground, a tennis court, or an extra classroom at school, and the community votes for it. And then, perhaps, the results can be measured. And those people who consistently vote for decisions that turn out to be more successful and in demand get proportionally more influence in subsequent votes. There are many interesting ideas here. I hear my economist friends actively discussing this. And it would be great if much more work were done in this direction. And then there's the philosophical side of the issue. Yes, jobs will change, and many other things will too, but perhaps by then the issue of thermonuclear energy will be resolved, and we will have an abundance of almost free energy. We will find ourselves in a post-scarcity world. What then will happen to money? Perhaps everyone will be better off. But what will happen to the sense of meaning? Because for many people, the meaning of life is tied to work and to providing for their families. And that's a very noble goal. So here many questions move from purely economic to almost philosophical. Are you concerned that people don't seem to be paying enough attention to this and not moving as fast as you'd like? What do you think needs to happen for people to realize the need for international cooperation on this issue? Yes, I am concerned about it, and I wish that in an ideal world, there would be much more cooperation right now, especially at the international level, and that there would be more research, discussion, and joint reflection on these topics. I am, frankly, quite surprised that it's not talked about more, considering that even by our estimates, the timelines are quite short. There are more aggressive forecasts, but even ours are 5-10 years, and for institutions and similar structures, that's a very short time to prepare. And one of my concerns is that existing institutions look very fragmented and do not have the level of influence that would be needed here. Perhaps there are simply no suitable institutions capable of handling these tasks right now. And if you add to this the current geopolitical tensions in the world, it becomes obvious that cooperation and coordination are becoming increasingly difficult. Just look at climate change and how difficult it is to reach any agreement on this issue. So we'll see. I think as the stakes rise and as these systems become more powerful, and perhaps also because they are already being integrated into products, ordinary people who don't work directly with this technology will start to feel the growth of its capabilities and power. And then it will reach governments. And perhaps, as we approach AGI, they will start to act more thoughtfully. Do you think some specific moment or incident will be required for everyone to pay attention? I don't know. I hope not. Most leading labs are acting responsibly enough. We try to be as responsible as possible. And as you know, if you've followed us all these years, this has always been at the center of everything we do. This doesn't mean we'll do everything perfectly, but we try to approach it as thoughtfully and scientifically as possible. I think most major labs are indeed trying to behave responsibly. Moreover, there is also very real commercial pressure that drives responsibility. Speaking of agents and the fact that you conditionally rent an agent to another company to perform certain tasks, it's important for the company to understand its limitations, boundaries, and safeguards, what it can do and what it shouldn't do, so as not to, for example, corrupt data or create other problems. So that's a good factor, because more irresponsible cowboy projects simply won't get orders. Large companies won't work with them. So I think the capitalist system can play a positive role here, reinforcing responsible behavior. But at the same time, there will always be bad actors. Perhaps individual states, organizations, or people who will build something on top of open source. Stopping this is extremely difficult, and then something can go wrong. One hopes it will be a medium-scale incident, a kind of warning shot for all of humanity. And perhaps this will be the moment when there will be a real opportunity to promote international standards or, at least, international cooperation and coordination at some basic high-level, to agree on minimum principles and rules that we are willing to accept. I hope that will be possible. Looking at the long-term perspective beyond AI and towards ASI, artificial superintelligence, do you think there are things that humans will be able to do, and machines never? I think that's the main question. And, as you know, it's related to one of my favorite topics, Turing machines. It always seemed to me that if we create AI and use it as a model or simulation of the mind, and then compare it with the real human mind, we will be able to see the differences and understand what exactly remains unique and special in human thinking. Perhaps it's creativity, perhaps it's emotions, perhaps it's dreams. There are many aspects of consciousness. There are many hypotheses about what can be computed and what cannot. And all of this comes down to the question of Turing machines. Where is the limit of a Turing machine's capabilities? For me, this is essentially the central question of my entire life. Ever since I learned about Turing and his ideas. I fell in love with this topic. It's my main passion. And I think that everything we are doing is an attempt to push the understanding of Turing machine capabilities to the limit, including things like protein folding. And frankly, I'm not sure where that limit is. Perhaps there is no limit at all. Of course, my friends in quantum computing would say that limits exist and that quantum computers are needed to model quantum systems. But I'm not so sure about that, and I've even discussed it with some quantum technology specialists. Perhaps we just need data from quantum systems to then create their classical simulation. And then we return to the question of the mind. Is it a completely classical computation, or is something else happening there? For example, Roger Penrose believes that there are quantum effects in the brain. If they really exist and consciousness is linked to them, then machines will never be able to have consciousness. At least, not classical machines. They will have to wait for quantum computers. But if these effects don't exist, then perhaps there is no limit at all. Perhaps everything in the universe is, in principle, computable, if viewed in the right way. And then Turing machines will be able to model everything in the universe. Frankly, if I were asked to bet now, I would bet on that. So, it turns out there's nothing that can't be done within such machines. Well, let's say, so far, no one has found anything fundamentally incomputable in the universe. So far. So far. Yes. And I think we've already shown that it's possible to go far beyond the traditional view of complexity theorists, like P = NP, and the idea of what classical computers are capable of today. Things like protein folding or playing Go are good examples. So I don't think anyone really knows where that limit is. That's what we're doing at DeepMind and Google. And that's what I'm trying to understand, where the limit is. But if you push this idea to its extreme, it turns out that here we are sitting, we feel the warmth of the light on our faces, we hear the hum of the equipment in the background, we feel the surface of the table under our hands. All of this, in principle, can be reproduced by a classical computer. Yes, I think ultimately that's why I love Kant so much. My two favorite philosophers are Kant and Spinoza, for different reasons. But Kant said that reality is a construction of the mind. And I think that's true. All those things you listed come into our sensory system and are felt differently: light, warmth, touching the table. But ultimately, it's all information, and we are information processing systems. I think that's what biology is. And that's how we will eventually cure all diseases if we view biology as an information processing system. And I believe that ultimately this will lead us to an idea that I've been pondering in my rare free 2 minutes, physical theories where information is the most fundamental entity in the universe. Not energy, not matter, but information. Perhaps, in the end, it's all interchangeable. We just perceive it differently. But as far as we know, all these amazing sensors we have can still be modeled by a Turing machine. That's why your simulated worlds are so important. Yes, exactly. Because it's one way to get there, where the limit of what we can simulate lies. After all, if you can simulate something, then in a sense, you understand it. I wanted to end the conversation with some personal reflections on what it's like to be at the center of all this. Tell me, does the emotional burden ever start to wear you down? Is there a sense of isolation? Yes. Listen, I sleep very little, partly because there's too much work, but also because I just have trouble sleeping. It's a very complex set of emotions, because, on the one hand, it's incredibly exciting. I'm literally doing everything I've ever dreamed of. And we are at the absolute frontier of science and applied science and machine learning. And it's intoxicating, as all scientists know, that feeling when you're on the edge of the unknown and discover something for the first time. And this happens to us almost every month, which is just amazing. But at the same time, of course, we, I, Shane, and others who have been doing this for a long time, understand the scale of what's coming better than anyone. And it's still underestimated. I mean, what will happen in about 10 years. Including philosophical questions. What does it mean to be human and what is truly important in that? All these questions will inevitably arise, and it's a huge responsibility. We have an amazing team that thinks about all of this. But for me personally, it's also something I've been preparing for my whole life. Since childhood, when I played chess, then computers, games, simulations, neuroscience, all of it led to such a moment. And in general terms, everything is happening roughly as I imagined. Perhaps that's what helps me cope. The feeling that I've been preparing for this. But have there been any moments that hit harder than you expected? Yes, absolutely, along the way, you know, even the AlphaGo match. Just seeing how we managed to crack Go. At a time when Go was this beautiful mystery, and it changed it. It was interesting and a little bittersweet at the same time, and even more recent things: language, images, and the question: what does this mean for creativity? I have immense respect and passion for the creative arts. I've done game design myself. I've talked to filmmakers, and for them, it's also a very interesting turning point. On the one hand, they have amazing tools that accelerate idea prototyping tenfold. But on the other hand, doesn't it replace some creative skills? So similar trade-offs arise everywhere. And I think that's inevitable when it comes to technology of such power and such transformative scale as AI, just as it was in the past with electricity or the internet. We've already seen that the history of human creation is the history of tool creation. We are beings who make tools, and we love it. And for some reason, we have a brain that is capable of understanding science and doing science, which is amazing in itself, and at the same time, it is insatiably curious. I think that's the essence of what it means to be human. And it seems I've had this virus from the beginning, and my way of answering these questions is to create AI. When you and other leaders in AI find yourselves in the same room, is there a sense of solidarity among you, that you all understand the stakes, all truly realize what's happening, or does competition keep you at a distance? Well, yes, we all know each other. I'm on good terms with almost everyone. Some of them don't get along with each other. And it's difficult, because we are simultaneously in the most fierce capitalist competition that perhaps has ever existed. My investor and venture capitalist friends, who were around during the dot-com era, say that things are about 10 times tougher and more intense now. And in many ways, I like that. I live for competition. I've always loved it, even since chess. But if you take a step back, I understand, and I hope everyone understands, that there's something much bigger at stake than just the success of individual companies and similar things. Looking at the next few decades, when you think about it, are there any big moments that personally cause you the most concern? I think that currently systems are, let's say, passive. The user invests energy, asks a question or formulates a task, and then the system provides some summary or answer. That is, everything is very much human-oriented. Human direction, human energy, and human ideas at the input. The next stage is agent systems, which I think we will see more and more. We are already seeing them now, but they are quite primitive. I think in the next couple of years, truly impressive and reliable versions will emerge. They will be incredibly useful and powerful if viewed as assistants or something similar, but they will be much more autonomous, which means the risks will also increase. So I am quite concerned about what such systems will be capable of, perhaps in 2-3 years. That's why we are already working on cybersecurity, preparing for a world where perhaps millions of agents will roam the internet. And what are you most looking forward to? Is there a day when you can retire knowing that your work is done, or is there more work ahead than can fit into one lifetime? Yes, I always, well, I could definitely use a vacation, and I would spend it doing science. Even a week of rest, even one day would be nice. But seriously, my mission has always been to help the world safely navigate AI through this frontier for the sake of all humanity. I think when we reach that point, then, of course, there will be superintelligence, post-AI, all the economic and social issues we've talked about. Perhaps I can help somehow there, but I believe the main part of my mission, my life's mission, if we manage this stage, is, of course, just a small task, just to help get it across the finish line, to help the world do it. It will require great cooperation. I am a fairly cooperative person, so I hope I can help with that. And then you can go on vacation. And then, yes, there will be a well-deserved creative vacation. Demis, thank you so much. Thanks for having me.