Transcription
All right, thank you all for coming. Um, today I wanted to step back from the day-to-day news in the newest models that are dropping or the rumors about drama at the labs and think about the bigger picture question of when we'll get to AGI, to artificial general intelligence, when we'll get to models that can do much of what you and I can do, at least when it comes to knowledge work.
Um, and the thing that's motivating this question is I have this podcast. I interview people on it to talk about this question. I've interviewed the CEOs of many of the AI labs, people like Dario, who was here earlier, and Demis, and the CEOs of the hyperscalers who are building out the compute to make this AI explosion possible, people like Mark Zuckerberg and Sati Nadella. I've interviewed the researchers, um, who are building these models, and economists who are thinking how to project their impact. And the range of possibilities, the range of timelines that these people anticipate is really big. Some of them think that AGI is two years away, others think it's 30 years away. And so I wanted to just think through where I think we are currently with AI and how far away we are from the transformative impact that people anticipate that AI could eventually have once it can do much of the work that humans can do.
So, um, before we get to what's missing in these models, we should note that they are truly incredible. Um, you can watch these models answer difficult problems for you, to code whole repositories for you, to reason. Uh, you know, we used to think reasoning is this thing that only humans can do, the pinnacle of human intelligence. And one of the first things these models cracked is the ability to just go away for five minutes, think hard about a problem, and come back to you with an answer. Um, so we should note that we've just made these magical systems, um, and done so really on a really rapid pace.
But um, I think it's it's also interesting that we can have something which feels so organic, passes a Turing test, and in terms of the actual economic value it generates is smaller than many other commonplace companies you would think of. Now, of course, the revenues of these companies are growing incredibly fast. Um, I think on the order of 3x a year. Uh, they're smaller than the amount of money that Kohl's or McDonald's makes in a year. Um, and Kohl's or McDonald's don't have AGI, right? So, I feel like once these companies do have AGI, their revenue should be way, way higher. And so, we had to explain why we can have systems which seem so smart when you talk to them, but um, are nowhere close to generating the kind of economic value that AGI implies. And even if AGI could just do white-collar work, so just interact over a computer, do the kinds of jobs you or I have, uh, all the knowledge work in the world, that's on the order of generating 30-40 trillion of economic value because that's the wages that are paid out to knowledge workers. So we're a thousandfold, at least a thousandfold away from that.
So what's going on here? Well, I think the big problem is that AI currently cannot learn on the job. There's no way for them to pick up information over the course of many sessions about you, your workplace, your preferences, um, how your company works. If you think about hiring a human, right? Like for the first three, six months, you're training them up. They're building up all this context. They're learning from their failures. They're practicing a task. It's not mainly their raw intellect which makes them valuable. It's this ability to learn continually on the job which makes them valuable. And that's why when you hire somebody, their experience at your firm ends up making them a lot more valuable just a few months in. LLMs currently just don't have this capability. You get the capabilities you get out of the box in the current session. But once that session is over, once you've been talking to for 30 minutes or an hour and you've exhausted the context memory, you're just, um, that whole memory, that tasted understanding it has of what problems you have, is totally gone, and you've got to restart from scratch.
And then people will suggest various ways in which you can fix this problem. They'll suggest, well, okay, maybe the session is lost, but you have feedback for it. You can just rewrite it back to the system prompt, or you can rewrite, um, its own feedback for itself back in the system prompt. You can like give it a bunch of context back into, uh, the system prompt. Um, I don't think this is enough to unlock the kind of learning that humans are actually experiencing on the job.
So here's an example. Imagine you're trying to teach a kid how to play the saxophone. Um, how you do that is you'd have them try to blow into one, and you see how it sounds, and they'd like, you know, practice a little bit, adjust, etc. Um, imagine this is the way it worked instead. You have a kid practice. You have a kid try once. The moment they fail, um, you send them out of the room, and then you write down on a piece of paper all the instructions about, here's what you messed up, here's how you should do it better next time, etc., and you call a totally different kid back in the room, and you have her just read these instructions and learn, try to play the saxophone without her having practiced this before. This is the kind of thing we're trying to do when we try to have the LLMs learn just based on text descriptions in their system prompt. And it's the only modality we as users have to teach these models anything. But just based on this example, you can see that it's totally insufficient to actually build up the kind of experience that people have to gain in order to do jobs.
Um, another solution people have is that we have longer context windows. So context windows for these models do get tend to get longer over time. Or maybe there's ways to summarize the context and then distill it back into the model again. Um, currently models have on the order of hundreds of thousands to a million tokens of context. Um, this, but this is only on the order of minutes to hours of information that it can keep track of. And, uh, even if you increase it to a couple of hours, that's still not unlocking the years-long level of context that you actually need to have to understand a domain, to understand how to solve problems. Um, and that's why I don't think this by itself will be enough either.
The other solution is that we now have RL working for these models. And so you might have noticed that people are talking about 01, 03, Deepseek R1. These are models that have been trained on solving verifiable math and coding problems. Um, and they become smarter as a result. They become much better at those domains. And you could hope that you could do similar things for your workplace. So whatever, uh, task they need to do, you could just reward them when they do it correctly, punish them when they do it poorly. And based on this, they'll build up an understanding of how to do your task. But when I think about the things I hire people for and whether this would be sufficient, I have editors who help me with my podcast. And the way they've gotten better over time is not because I've built bespoke environments for them where they're practicing one subtask and then I give them a reward if they get like a lot of views on YouTube, and another subtask, and if they make a really good animation, they get a reward. It's just that there's a much more seamless process where they are noticing things themselves. They are observing what works with the audience, what preferences I have, um, how to improve their day-to-day workflows. And this seamless organic process just does not correspond to the method of RL fine-tuning that you could use right now to make one of these models better for yourself through the fine-tuning API.
And so there's this broader question you can ask about why is it the case that, um, the Fortune 500 isn't readily adopting AI to automate all their workflows. If you look at non-coding use cases for enterprises, um, the amount of LLM adoption is actually surprisingly low. And so what's going on here? Well, sometimes people will suggest that these people, these companies, just haven't, they're too stodgy. The management isn't thinking enough about how workflows can be automated. Um, and if they just did a couple basic things like build up little scaffolds where the LLM has a harness, it has external memory that it can write to, it can read all your files, um, that will be enough to unlock significant economic gains already. Or if they give it more training data, um, about how things at their company work. Uh, or we just like got more training data about how different jobs in the world work and the companies, the LLM companies use that to train these models better, um, that would be enough to just let these models be AGIs. And I think neither of these techniques will be powerful enough for the same reasons that we just discussed before, that there isn't some training paradigm which allows you to get all this tacit information back into the model. RL fine-tuning isn't it. And text-on information does work for coding, so we have noticed coding models that are incredibly good, um, but that's because coding has this external scaffold of memory, which is the codebase itself. And also the whole task is very explicit, right? You're writing to a file. You're writing the explicit logic involved in solving the problem to a file. Most jobs in the world are just not like this. There is no external scaffold of memory that is built in. And the information about how to do the task doesn't exist as these huge GitHub repos that you can train on, um, and requires much more context and information about specific companies, specific people, specific workflows, uh, specific preferences. And this will just be much harder to solve than coding ended up being.
Um, now I do think, uh, we will eventually get to AGI, right? So this is a bottleneck to AGI. We've had many other bottlenecks to AI becoming better over time that we've solved. It's worth remembering that deep learning at scale, um, has only been a thing for 10-15 years, and already we have incredibly capable systems. If you just project out forward what this progress can mean, eventually there will be a solution to this problem. And I think this will be much more powerful than people are anticipating with words like AGI, because if you have a system that is capable of learning on the job, it's significantly different from what humans are doing. So if you and I are learning how to do our jobs, we are learning, um, how we do one job at a time. With an AI, because the model can learn from everything any of its copies are seeing, you could have a single model that is broadly deployed to the economy. It's doing all the different kinds of jobs there are, and that one model is learning from the experience of all of its copies. So you have one model that's basically learning how to do every single job in the economy. It's basically learning anything you or I know, but at the same time. And that, I feel, is functionally superintelligence. If you do solve continual learning robustly, then you would have a system which is, um, becoming enormously more capable than any one human being by itself. And this doesn't involve any sort of superintelligence algorithmic breakthroughs. It just requires this capability humans already have, plus the natural endowments, um, or the intrinsic endowments that digital minds will have, which is the fact that they can learn from each other's experience much more seamlessly.
Um, okay, so what actually are my predictions? When are we going to unlock these key capabilities? Um, so one thing that's needed beyond continual learning is computer use, um, for these models to automate or help you with significant amounts of knowledge work. It's not enough that they're just chatbots that you ask questions to at chatgbt.com. They actually need to be able to use your computer, um, and to in order to use the applications you commonly use, in order to do many hours of work on your computer. This is what knowledge work involves. And this big bottleneck, unlocking this bottleneck, I don't think is just a year or two years away. There are significant challenges to training a model which can do these things. You have much longer agentic tasks which require different kinds of data, video, video information, much longer rollouts because you had to observe its, uh, ability to perform the task over many hours. It's not just like language where you can train on every single token. And we don't have this existing corpus that we had with LLMs of internet text data which you can just shove into these models to just automatically make them better. This data has to be manually collected. People have to, you know, rehearse tasks or build environments for the LLM to rehearse tasks for practicing common pieces of software. Um, and it just takes a while to build the algorithms to make this kind of training possible, to make this kind of data collection possible. Um, and for these reasons, I think this will be a significantly harder problem than many people anticipate.
Um, so in terms of where we are probably at in terms of computer use, I think it, you know, it's, it's a probably a good analogy to say it's like where we were with GPT-2 for LLMs. And GPT-2 was released, I think, five, six years ago. There were four years between GPT-2 and GPT-4. And GPT-2, you can think of as, here's something which, if you really steer hard at it, um, can do some cool things, has some cool demos, you can like try to project it forward and see like, if you really believe this is a really good indication, something cool will, um, uh, and we have that with computer use. We have tools like Operator, which can start using common pieces of software, um, but there will be many more years of collecting enough data to make this a robust, viable product. I think in many ways it's harder than scaling up LLMs because, uh, from GPT-2 to GPT-4, you could just make the model bigger and just throw more of the internet text at it. We already had that internet text collected. With computer use, we'll have to collect this data manually. That will be much harder. On the other hand, we have a lot more compute in the world. AI research has advanced. There are more AI researchers in the world. So, I think those even out and maybe there's a similar gap between these two. So, four years from now seems like a reasonable guess to having some kind of reliable computer use.
Okay. And I think this is like, why is computer use important? Well, vaguely, it's involved with knowledge work. But for me, when I'm thinking about applications, I need, I want an AI that can do my taxes for me, right? I want something that can go trace down all the receipts, um, go hassle people for invoices, spend a week just going through my Amazon orders, what's a business expense, what's not. Um, uh, and that kind of week-long task, which requires using all kinds of different interfaces, filling out forms, going through, um, going through different places I've made orders, will require this kind of computer use. And I think that's three, four years away. I expect this kind of capability in 2028.
And then this continual learning thing, um, I think will be, uh, plausible by 2032. And on the one hand, that's a long way away from now. On the other hand, remember, if continual learning is solved, we don't just have AGI, we have this sort of superintelligence boom from one model learning how to do every single task in the economy. Um, so the whole vibe of this talk has been kind of that, oh, you know, AI is further away than some people might think. Dario was just speaking earlier. I'm sure he said that it's, you know, we're, we're right on the cusp. But it's worth remembering the predictions, uh, that I think are plausible here are just incredibly wild, right? Being seven years away from a model that is just functionally becoming superintelligent by learning how to do every single job in the economy, at the very least, is automating much of knowledge work, um, generating tens of trillions of dollars of revenue, or at least generating tens of trillions of dollars of economic value. That's just a really wild thing to happen in just a few years. And in the meantime, of course, there will be a ramp-up. So this isn't just happening like that. In the meantime, we're unlocking more and more capabilities. Um, so I'm anticipating a pretty wild next few years. Um, and this is just to illustrate why I think seven years is a plausible timeline, which sounds like not that much time. But if you, GPT-C1, which is just this idea that you could train a transformer on language and it might have meta-learning, it might have important capabilities, was come up with seven years ago. Uh, if I certainly wasn't paying attention to it at the time, I'm sure you weren't. I think it's possible that we are in a similar position with continual learning, that people are trying out different ideas, coming up with the paradigm that might be scalable in seven years' time, and that's why I think 2032 is a reasonable timeline.
Um, now I, I mean, one interesting wrinkle here is that, uh, it becomes hard to forecast AI progress beyond the 2030s or early 2030s because, uh, right now, the way AI systems are getting better is by throwing tremendously more compute into them year after year. Frontier models are showing down 4x more compute on average than the year before. And if you understand exponentials, you just realize this is not a tenable thing over the course of a decade, right? At which point you're doing millions of times more compute to train a system than the decade before. Um, and so if you just, my friend Romeo Dean and I were just playing around in spreadsheets and we just wanted to see if current trends in compute scaling continued, what fraction of world energy or world GDP would be going towards training frontier models or to deploying frontier models. And here we're looking at energy. Um, I'm not saying this is, I'm not saying this is actually likely. Rather, the, you know, here we have 11 terawatts by 2040, and that's way more than the, that's like two, three times more the entire world produces in electricity cumulatively. I'm not saying that will actually happen. This is more to illustrate the fact that it will be very hard to make happen if we don't have AGI, and it's unlikely to happen if we haven't made significant progress towards AI, um, to begin with. So that's, this is all to say that if we don't get AGI by 2030, 2035, I think it's very plausible that AGI is then decades away, because then you just have to do these old-school algorithmic breakthroughs. You can't just keep scaling the compute, um, which has been the main way the AIs have been getting better.
All right, so where does that leave us? Um, what does that mean for us? Well, we already have models that are incredibly powerful that you can use for many applications already. I just think it's wild that we have models that can reason. Again, this is the thing that we thought was the pinnacle. Um, you know, if you read Arthur Staler or something, he'll say, uh, reasoning is a thing that sets humans apart from other animals. And it's like the first thing we cracked with these LLMs. Um, you can just ask Gemini 2.5 Pro or 03 or something a question and have it go reason for a while, and then just combine a bunch of information, understand your preferences, think through what you're trying to build, uh, and come back to you with a really smart answer. Um, and, uh, I, what it, I think what it means for applications that one might want to build is that because there are significant bottlenecks that have to be unlocked before we get to AGI, um, that will open up many opportunities to build compelling products that will only be viable once those capabilities are unlocked. So to give you one example, um, the company behind Cursor was, uh, uh, started in 2022. The, um, it was only last year that we had coding models that were capable enough to make LLM coding a really compelling product. And so, uh, Cursor hits product-market fit after Claude 3.5 Sonnet is launched. Um, and so what we had for coding, I think we'll have for many other things, right? Computer use will be unlocked. Think about what kinds of startups or what kinds of products within big companies will be made viable once that happens, once continual learning is unlocked. There's going to be many other things like this which will open up huge opportunities for different kinds of ideas, different kinds of startups, different kinds of businesses.
Um, and I remember, so I moved back to San Francisco about two, three years ago, and at the time I was doing the podcast as a side project. My main goal was to build some kind of GPT wrapper, some kind of startup. Um, and I remember thinking at the time that I'm quite late to everything already, like, uh, GPT-3.5 has come out, you know, all the AI startups were made, and this is kind of hopeless. And of course, um, now we realize that there were many more AI companies to be made. Of course, I don't think I was the right person to make them, but, um, but I think we'll, similarly, in a similar position, uh, in a few years, where we'll be like, right now, we're actually still quite early in AI. There's going to be way more capabilities that are unlocked, and those capabilities make viable way more products, way more ideas, um, way more different ways of interfacing with these AIs.
I think what it means for us now and how we can use AIs right now to improve, um, our work, to make ourselves more productive, um, is the following. So I've been trying a couple of techniques myself that have helped me tremendously and, um, uh, saved me a lot of time, made me more productive. The biggest one is that I have this huge Google Doc of 20,000 words that I keep compiling information that is relevant to me, uh, that, you know, basically things like, of long-running journal of logs of like problems I'm facing at the business or with the podcast, different problems I'm trying to brainstorm, um, uh, different ideas I've come up with, meeting summaries. I have AI tools which summarize meetings I'm in, and then I'll just copy-paste them into this document. Um, even things like common email templates I send out, for example, to guests or to, um, or prompts I have for coming up with clips, writing common copy. Um, and this is, you can think about what the relevant part of this is to you. Basically, I, the way I would think about this is whatever onboarding document that you would give to a new employee at your company, every single thing in there, just shove it into one document, and then add it to the LLM at the beginning of any session where any of that information might be relevant to it. LLM context windows are not big enough to do many year-long continual learning, um, kinds of tasks, but they are long enough to integrate significant amounts of onboarding information, right? They can, hundreds of pages worth of information they can consider in every single session. So, um, this is actually even not possible with humans. A human can't just read a hundred pages of information about your company, your preferences, your workflows, common things you've done in the past, and consider that before every single answer you give them. These LLMs can. I would just make use of the capability and make this kind of document.
The other big thing I would do for the podcast is I'm interviewing people across a wide range of disciplines, people who are bio-researchers or AI researchers or thinking about energy or physics or, um, many other fields, and it's quite important to me to be able to learn fast. There is a lot of interesting research in terms of how you can speed up learning. There was a researcher, um, Benjamin Bloom, who discovered that one-on-one tutoring, um, has two standard deviations better outcomes for learning than learning in a classroom with many other students. Now, this isn't really a relevant or practical piece of information for most of us until now, because it's, you can't get a one-on-one tutor for whatever field you want to learn about at a moment's notice. You can use LLMs for this purpose. So, the thing I like to do is ask the LLM, look, you're a Socratic tutor. I don't want you just throw information at me, just explain this concept to me by writing a long monologue. I want you to tutor me. I want you to be a tutor. And what that involves is, um, don't explain the whole concept to me. Um, keep just motivating questions which help me understand subconcepts and help me grow the full picture, and don't move on until you've asked me enough questions that you're satisfied that I have understood this topic, um, to your satisfaction. And this has been a tremendously huge unlock in helping me better understand topics. It's really like having expert tutors across every single field at a moment's notice. But you do have to use the tool properly to get this capability out. You can't just have it, um, spit Wikipedia text at you, which is the default way in which it gives you information. You have to prompt it appropriately.
And finally, um, I think it's important, uh, given how fast AI is progressing, to not just wait for tools that are handed down to you by your organization. This can be quite a slow process. Um, in many cases, they're handing you tools that are already outdated. And there are so many AI tools that they might not have procured the one that is most relevant to your workflows or to the problems you're facing. These tools that are in many cases free, um, in other cases not that expensive, on the order of $20 a month. Um, if they can speed up your productivity, I would just really recommend experimenting yourself beyond the tools that are just handed down to you.
Okay. Um, thank you guys so much. [Applause] [Music]