Transcription
What happens in the weeks, months, and years after we achieve AGI? What happens to the economy? What happens to people's ambitions? All of these questions I've been thinking about, and I really haven't talked much about it on this channel. So that's what I want to talk about today.
I found this really great article on the Less Wrong blog, and I want to go over it with you and give you my thoughts on it. So let's get right into it.
Here's the blog: "By Default, Capital Will Matter More Than Ever After AGI." It just came out a few days ago and it's by L. Rud Rol. I couldn't find much information about this author, but I'll drop their links that I did find in the description below.
Now, to set the scene, a lot of people think once AGI is truly achieved and at production scale, we will effectively have infinite resources because we'll have infinite thought labor and infinite manual labor through artificial intelligence and humanoid robots. He gets into really deep economic topics here, and so I'm going to try to explain it as best I can.
He starts by saying, "I've heard many people say something like money won't matter post-AGI." This has always struck me as odd and most likely completely incorrect. He makes really strong points as to why that's the case, and I've heard Dave Shapiro, who has talked about post-AGI economics on his channel quite a bit, call out this same thing.
First, he defines what labor actually means. Labor means human mental and physical effort that produces something of value. So whether you're producing something on a factory floor or you're a content creator like myself, an engineer, whatever that is, capital goods are things like factories, data centers, and software—things humans have built that are used in the production of goods and services.
Now, here's the crux of his argument: the key economic effect of AI is that it makes capital a more and more general substitute for labor. This means if you have this entity which you can essentially just pay for infinite labor, both mental labor and physical labor, then the capital itself—the money—is the only thing that matters at that point. Human labor is not important because you can just choose to pay AI to produce the same value as a human would.
There's less need to pay humans for their time to perform work because you can replace that with capital, basically paying AI. This is a really big idea here, and this is something that we're going to have to contend with in the very near future.
He says labor replacing AI means human labor is what he means. The ability to buy results in the real world will dramatically go up. Human ability to wield power in the real world will dramatically go up, at least without money, including because there will be no more incentive for states, companies, or other institutions to care about humans.
By the way, by reading this, I'm not saying I agree. I'll try to give you my thoughts, but I'm also trying to figure out where I stand along with you. I'm trying to understand his point from many different perspectives on these issues.
He argues that societies become static—permanently static—and whatever position in life that you've had in the past, it'll be really hard, if not impossible, to move up from there. Essentially, your current capital is your value. Current power imbalances might be amplified and turned immutable.
But he says this isn't about just surviving. Given sufficiently strong AI, this is not a risk about insufficient material comfort. Just the everyday things that people will need to have, governments could institute UBI with the AI-derived wealth.
So we're going to have all of this artificial intelligence essentially producing infinite resources, and from those infinite resources, governments can institute UBI—Universal Basic Income—essentially just paying everybody to live.
Now, I'm still trying to figure out how I feel about UBI in general. I feel two ways: one, it makes sense if you have infinite value created by AI; you should distribute it just to let people live. But at the same time, when you do that, something tells me that human innovation and desire to create will go away.
He goes on to talk about even if the United States alone is the sole winner of the artificial intelligence race and captures all of its value, he believes that there will be enough wealthy people who care about the rest of the world in the United States to end material poverty simply by the fact that they will be giving away money to everybody in a kind of "rising tides lift all boats" scenario.
And before we're in the post-AGI world where seemingly nothing matters, let me tell you about something that really matters: the partner of today's video, Vulture. Reap the benefits of the world's largest independent cloud provider. When you bring your GPU workloads over to Vulture, they have the latest Nvidia GPUs spanning 32 locations across six continents. Vulture delivers industry-leading price to performance and serious accessibility and reliability.
Vulture's global, fully composable cloud infrastructure moves your application closer to your users and frees you from vendor lock-in, allowing you to bring your own networking and database solutions. If you need to scale beyond just a single cluster, Vulture's Kubernetes engine allows you to take full control over your deployment, offering up a 100% free control plane.
So if you're tired of waiting for GPUs from other providers, make sure to use Vulture. You can deploy at any scale immediately, and they have H100s, L4s, and more available right now. You can get a fraction of a card or fully dedicated bare metal systems, which gives you full control over your hardware and your throughput.
They also have one-click installation of all the applications you might need for advanced machine learning workloads, allowing you to get up and running in minutes, not hours. So experience the Vulture difference. Don't get bogged down by severe wait times or limited locations. Try it free today with a $300 credit for your first 30 days when you visit getvulture.com/Burman and make sure to use code Burman300 at checkout to get that credit. Thanks again to Vulture for sponsoring this video.
Now, he actually put this section in parentheses, and I found it fascinating. Let me read it to you: "If 1% of American billionaire wealth was spent on wealth transfers to foreigners, it would take 16 doublings of American billionaire wealth, as expressed in purchasing power for human needs—a roughly 70,000 times increase—before they could afford to give $500,000 equivalent to every person on Earth. In a singularity scenario where the economy's doubling time is months, this would not take long."
So kind of crazy numbers to think about, but again, if we're talking about AGI, if we're talking about the intelligence explosion, these numbers might not be that crazy. But he counters with, if the intelligence explosion is not singularity-like—meaning it is not completely explosive—then there's a chance that just material comfort might actually be an issue.
So he's really betting on the fact that the intelligence explosion will be an explosion. He goes on to say, and I agree with this, what most emotionally moves me about these scenarios is that a static society with a locked-in ruling caste does not seem dynamic or alive to me. We should not kill human ambition if we can help it.
Now, he thinks human ambition will be killed by the fact that if you can't move up in the world, why would you have ambition? And you know what? Of course, that makes a lot of sense. But I think there's another angle to which human ambition can be killed: if there's no need to create, if there's no need to move up in the world, if you just have everything you ever needed, why would you have ambition? Why would you want to create something new and really make huge sacrifices to yourself, your life, your health to create something new for the world, which many people do right now?
Next, he talks about what he calls the default solution. He starts by saying, "Let's just assume that essentially all human mental and physical labor has no more value or very trivial value, essentially because AI can do it better, whether that's robots or digital AI."
He says, "Okay, well, there are two levels in the standard solution: one, governments will adopt Universal Basic Income, so essentially there's infinite resources, infinite value created by AI, so we're just going to pay everybody to live; and two, we will quickly hit superintelligence, and assuming the superintelligence is aligned—big assumption, of course—we'll live in a post-scarcity technological wonderland where everything is possible."
Yeah, that's what I want. I tend to be very optimistic. I'm hoping that is the outcome. I think it'll probably be something like that. I also don't want to seem naive and just think that everything's going to work out and we don't have to work at it.
He goes on in this scenario to say money will continue to be a thing unless we have one single AI system doing all economic planning, which is possible. Prices are largely about communicating information, so if there are many actors and they trade with each other, the strong assumption should be that there are prices—that there will be prices—even if humans do not see them or interact with them.
Of course, if there's any kind of trading, there's a market. If there's a market, there's prices. So remember too that however sharp the singularity, abundance will still be finite and must therefore be allocated.
Now, I don't know if abundance will still be finite if we end up leaving Earth. Obviously, if we're constrained by Earth's resources, then yes, abundance—even extreme abundance—will be finite. But if we start being a multiplanetary species, then maybe it won't be finite; it will be infinite.
Next, he argues something which I never actually explicitly put into terminology like this, but I dealt with quite a bit while building a startup. His argument is money currently struggles to buy talent. Meaning, even if you have all the money in the world, the best people at whatever the job is aren't always necessarily driven by whoever is going to pay them the most.
That was definitely the case in Silicon Valley. The top companies in Silicon Valley—Google, Meta, Apple, Microsoft—they all pay top salaries. They have comfy packages; you get all the amenities you can ever imagine—massages and free food every day, your laundry is done for you, gyms on campus. But a lot of those companies are losing that top talent to people going to start their own startup or joining very early startups.
That's because for some people, just getting paid the most is not enough. They want to be part of a mission. They want to be part of a purpose. They want to work on the coolest, hardest problems with other incredibly smart people. These larger companies sometimes slow down; they often run into the innovator's dilemma.
His point is well received by me; I agree completely. So listen to his example: consider Blue Origin versus SpaceX. Blue Origin was started two years earlier, had much better funding for most of its history, and even today employs almost as many people as SpaceX. Yet SpaceX has crushingly dominated Blue Origin.
In 2000, Jeff Bezos had $4.7 billion at hand, but it is hard to see what he could have done not to lose out to the comparatively money-poor SpaceX with its intense culture and outlier talent. So there is something to be said about the culture. The culture typically starts from the top; the top is Elon Musk.
If you want to convert money into results, the deepest problem you are likely to face is hiring the right talent. Couldn't agree more.
Here are the issues with getting the top talent, all of which I've experienced:
1. It's often hard to judge talent unless you yourself have considerable talent in the same domain. This is critical if you are going to start a company, especially in the software space. You better be an expert at whatever you're starting.
For me, it took learning to code because I found that I was just relying on somebody else to build whatever my idea was, and I couldn't always trust that this was the best code or they were moving at a speed that was satisfactory to me. So I just learned to code, and then all of a sudden, when I learned to code, I was able to hire other people who I could clearly see were much better than me at coding.
2. Talent is rare. And it's not only that talent is rare; they're also being pulled in a thousand different directions. They have every option under the sun, and even if you can locate the top talent, the top talent tends to be less amenable to being bought out by money than others.
Very true. This is all very well-known information in Silicon Valley, but with labor-replacing AI, these problems go away, and that is a big problem.
First, you might not be able to judge AI talent. Even the AI evals ecosystem might find it hard to properly judge. Evals are hard. But the real difference is that AI can be cloned. Currently, huge pools of money chase after a single star researcher who's made a breakthrough and thus had their talent made legible to those who control money.
But the star researcher that is an AI can just be cloned. So that's the difference between humans and AI: you can just clone the best AI. So everyone—or at least everyone with enough money to burn on GPUs—gets the AI star researcher.
This is the intelligence explosion part of the situational awareness paper, where as soon as AI starts being able to do cutting-edge research and then applying that research to itself, that's when we have the intelligence explosion. Because we can essentially infinitely scale up these AI researchers, and that means if you have the most money, you can simply buy the most GPUs and clone the best AI.
This is his point: this is why he says in this post-AGI world, things become much more static. Then, of course, the price of talent will go down massively because the AIs will get cheaper than the equivalent human labor, and because competition will be fiercer because the AI can be cloned.
He goes on to say the final bottleneck for converting money into talent is that people have preferences—usually complex preferences. It's not enough to just say, "Well, this company's paying me $300,000 a year; this other company is paying me $31,000 a year, so obviously I'm going to go with them." There are many more factors that go into a decision—too many to even list right now.
Oftentimes, the best talent, as I mentioned, is following expertise, passion, a mission. They want to work with the best talent. But AI is not that way. You simply pay the most, and you get the best AIs.
AI exists specifically so that they can be trivially bought out, at least within the bounds of their safety training. The genius AI mathematician, unlike the human one, will happily spend its limited time on Earth proving the correctness of schlep code.
Eventually, AI is just going to be better than every human. So if AI is getting cheaper and cheaper, better and better, then capital is really the only thing that matters. Because whoever can buy the most AI will get the best results.
Now we start getting into what makes people tick. Most people's power leverage derives from their labor. I could say that's the case for me. It's not only my power leverage, but it's also my identity. I'm trying to make that less so, but in the past, especially when I was running companies, my identity was so closely tied to the thing that I was doing—my work.
It wasn't actually very healthy for myself, and I'm again trying to get out of that. But still to this day, what makes me tick is what I do for work. Labor-replacing AI also deprives almost everyone of their main lever of power and leverage.
Most obviously, if you're the average Joe, you have money because someone somewhere pays you to spend your mental and/or physical efforts solving their problems. But wait, we assumed that there's UBI—problem solved, right? Maybe not.
Now he starts to talk about why states take care of people—states being governments. UBI is granted by states that care about human welfare. There are many reasons why states care and might care about human welfare, and this has been more of a recent phenomenon, or in the last few centuries.
The reasons why they started to care more and more about human welfare include moral changes downstream of the Enlightenment, in particular increased centering of liberalism and individualism, affluence in technology, and incentives for states to care about freedom, prosperity, and education.
But again, AI might break this model. AI will help a lot with the second point, yes. It will have some complicated effect on the first, yes. But here I want to dig in a bit more into the third because I think at this point it's underappreciated.
Since the Industrial Revolution, the interests of states and people have been unusually aligned. A strong state needs efficient markets, a good education system that creates skilled workers, and a prosperous middle class that creates demand. It benefits from using talent regardless of its class origin.
It also benefits from allowing high levels of freedom to foster science, technology, and the arts and media that result in global soft power and cultural influence. So this is just setting the scene as to why states have been aligned with human interests, at least up till this point.
He goes on to say, but with labor-replacing AI, the incentives of states—in the sense of what action states should take to maximize their competitiveness against other states and/or their own power—will no longer be aligned with humans in this way.
He compares it to the time of feudalism. He says during feudalism, the incentive was to extract as much as possible from the peasants without them dying. After labor-replacing AI, humans will be less a resource to be mined and more just irrelevant.
God, that is a scary thought! However, spending fewer resources on humans and more on the AIs that sustain the state's competitive advantage will still be incentivized, and that is where the loop closes. That is the really potentially bad outcome: humans will have less leverage over states because they're less valuable, and their incentives are less aligned with the states.
So of course, they're going to have less leverage over the states once AI can do all the labor that keeps the economy going and the military powerful. The state has no more reason to care about the demands of its humans.
Okay, so now he's going to illustrate why he believes that there are not going to be any more outlier outcomes. What does that actually mean? Outlier outcomes mean somebody who didn't necessarily start with money or power figured out how to create a lot of value and then all of a sudden becomes an insider with money and power.
Whatever their perspective that they brought from the outside, that is the major change that people typically see in the world. But of course, we already know where this is going. If people aren't able to make these massive improvements in their status in life, then how will we see change in the world?
They're not moving up; they're not bringing new ideas to the inside, and thus we're not going to see major idea changes. He gives a few examples of outlier success, and one is entrepreneurship—something that is obviously near and dear to my heart.
I found this really interesting. I'd never heard this before: entrepreneurship is increasingly what Matt Clifford calls the "technology of ambition," which I absolutely love that phrase. Entrepreneurship has become easier. AI tools already make small teams much more effective without needing to hire new employees.
This is very true. There have been many predictions that we're going to see the first billion-dollar company with just one person, and I think that's great. That doesn't necessarily mean we're going to hire fewer people. In my mind, that means we're going to have many more startups.
They also reduce the entry barrier to new skills and fields. However, labor-replacing AI makes the tenability of entrepreneurship uncertain. There is some narrow world in which AIs remain mostly tool-like, and entrepreneurs can succeed long after most human labor is automated because they can provide agency and direction.
However, it also seems likely that sufficiently strong AI will by default obsolete human entrepreneurship. That is horrible! I really hope that never happens.
For example, VC funds might be able to directly convert money into hundreds of startup attempts, all run by AIs, without having to go through the intermediate route of finding human entrepreneurs to manage the AIs for them.
Let's pause for a second and think about what the five levels of intelligence are as defined by OpenAI. Here it is:
- Level 1: Chatbots—thoughts AI conversational language (that's what we have right now).
- Level 2: Reasoners—human-level problem solving (we kind of have that now, especially with the 01 and O3 models).
- Level 3: Agent systems that can take actions (we're kind of right in the middle or on the verge of reaching that).
- Level 4: Innovators—AI that can aid invention (that's what we're talking about in this article).
- Level 5: Organizations—AI that can do the work of an organization (an entire organization that is created and run by AI).
So imagine you are a VC. You have all of the capital; you simply hire a team of AIs to create new ideas, to create innovation, and all of a sudden, you don't need the human entrepreneurs anymore.
Another area of outlier success is the hard sciences. Now we've referred back to the fourth level in that chart that I just showed you. If AI is able to be on the cutting edge of science and math and is actually discovering new science and new math, then we would just scale up that AI, and it would be able to discover new science and new math much more quickly than any human scientist or mathematician would.
At that point, that is kind of made obsolete as well. Same with intellectuals; it's literally the same category. He does say that politicians might be one of the least affected options, which kind of makes me sad. I'd guess that most humans specifically want a human to do that job.
Because politicians get to set the rules for what's allowed, he goes on to talk about military, religions, and a few other things, but I'm going to skip over those. If you want to check out the full article, please do. I will drop the link in the description below.
All right, so what happens here? If we are in this permanent caste system and nobody can move up, then how do we actually fix it? Well, he thinks enforced equality is unlikely.
So imagine that labor-replacing AI is here, UBI is passed, so no one is starving. There's a massive scramble between countries and companies to make the best use of AI. This is all capital intensive, so everyone needs to woo holders of capital. The top AI companies wield power on the level of states—kind of maybe already seeing a little bit of that.
The redistribution of wealth is unlikely to end up on top of the political agenda. Now here's the key argument: even if the future is a glorious transhumanist utopia, it is unlikely that people will be starting in it at an equal footing.
Due to the previous arguments, it is also unlikely that they will be able to greatly change the relative footing later on. So what are the default outcomes?
So grant the assumptions at the start of this post and the above analysis. Then the post-labor-replacing AI world involves money being able to buy results in the real world better than ever. People's labor gives them less leverage than ever before. Achieving outlier success through your labor in most or all areas is now impossible.
There is no transformative leveling of capital, either within or between countries. So if you have money, you're going to be able to make more money. If you don't have money, it's going to be difficult, if not impossible, to move up and gain more money.
Since AI is smarter, more efficient, and cheaper than human labor, then that's what is going to be used. So humans will have less and less leverage because there's not much incentive for you to have an outlier success, and really a bunch of things are preventing you from having outlier success.
That's not going to be much of a thing anymore, and most likely there's not going to be any leveling of capital.
So what are the best cases and worst cases as predicted by this author? In the best case, this is a world like a more unequal, unprecedentedly static, and much richer Norway—a massive pot of non-human labor resources, oil. AI has benefits that flow through to everyone, and yes, some are richer than others, but everyone has a great standard of living.
So he basically just compares it to a big Norway. The only realistic forms of human ambition are playing local social and political games within your social network and class. If you don't have a lot of capital, you don't have a lot of chance of affecting the broader world anymore.
The worst-case scenario: AI trillionaires have near-unlimited and unchecked power, and there's a permanent aristocracy that is locked in based on how much capital they had at the time of labor-replacing AI. The power disparities between classes might make modern people shiver, much like modern people consider feudal status hierarchies grotesque.
I found all of this to be absolutely fascinating. Again, I'll drop the full link to the article down below.
Now, there's one more thing I want to talk about because he doesn't necessarily touch on one last point. Let's say everyone gets infinite resources; they essentially get all the money they would ever need, and they no longer have to work. That is a possible outcome from AGI.
I want to show an example of a post that I came across just a day ago. So V. Houth, who was the founder of Loom, which was acquired, said, "I am rich, and I have no idea what to do with my life."
I talk about leaving Loom, giving up $60 million, laring as on breaking up with my girlfriend, insecurities, a brief stint at Doge, and how I'm now in Hawaii self-studying physics.
Now, I read this whole thing. I'm not going to do that right now; I'll drop it down below. But basically, what this post is about is this person essentially all of a sudden had a tremendous amount of wealth to the point where he no longer has to work ever again in his whole life.
He thought he knew what to do after that. He started to start a new company around robotics but then just realized, "I'm not that into it." He continues to just not be that into things, and he broke up with his girlfriend and moved to Hawaii and now he's self-studying physics.
But here's the thing: when you all of a sudden have infinite resources—essentially infinite resources—where does your ambition come from? Where does that drive come from?
I think maybe some people will just have that drive inherently, but again, if everything is solved, if there are no hard problems out there anymore that AI can't do better than you, what are we going to do as humans, especially the most ambitious humans?
I actually tagged Dave Shapiro in this to get his thoughts. So I'm going to finish on that. I said, "What are your thoughts on this one? It's no surprise that someone in their prime wants to find something impactful to do. One of our primary evolved traits is to maximize our social status, and in a world rich with possibilities, this program will activate. All the hallmarks are here: a bigger, more impactful mission emerges."
So that is what he started to do, but basically wasn't able to do. The second point is maybe he's just writing about all of this to flex or to tell people about his story—who knows?
But his final point is always make interesting choices. Act like you're the main character of your own story. Do whatever you want. Live your best life.
And that I can agree with. If you enjoyed this video, please consider giving a like and subscribe, and I'll see you in the next one.