Transcription
I would say the rate of progress is exponential right now on at least three key dimensions.
So number one is the algorithms themselves. The models are improving at a rate of, I don't know, three to four times a year. They're not just getting faster and better. But qualitatively, they're different. Remember, we started with pure LLM chatbots. Then we went to reasoning models. We didn't even get to the agents part of it yet, but that's the next big leap after reasoning models. We're just starting to scratch the surface there.
Then you've got the chips. Depending on how you measure it, each generation of chips is probably three or four times better than the last. It's not just the individual chips that are getting better. They're figuring out how to network them together. Like with NVL72, it's like a rack system to create much better performance at the data center level. And that would be the third area where you're seeing basically exponential progress. Just look at the number of GPUs being deployed in data centers. So when Elon first started training Grock, I think they had maybe 100,000 GPUs. Now they're up to 300,000. They're on the way to a million. Same thing with OpenAI's data center, Stargate; and within a couple of years, they'll be at, I don't know, 5 million GPUs, 10 million GPUs.
The algorithms, the chips, and the data centers are all improving or scaling at a rate of, I don't know, 3 to 4x a year. That's 10x every 2 years. Where people don't understand exponential progress is that if you're getting better at 10x every 2 years, that doesn't mean you'll be at 20x in 4 years. It means you'll be at 100x. So you multiply those things together, the algorithms, the chips, and then the raw compute that's available, you're talking about a millionx increase. Some of which will be captured in price reductions, some of it will be in the performance ceiling, and then some of it will just be in the overall amount of AI compute that's available to the economy. But the impact of this thing is going to be absolutely massive. And I think people still don't even appreciate that fact because they don't understand exponential progress.