Transcription
Last Friday, the US government did something extraordinary. It effectively forced one of America's leading artificial intelligence companies to withdraw its most advanced product from the market. Anthropic, the maker of the frontier AI model, Mythos, and its commercially available cousin, Fable, have been given little warning and according to reports, roughly 90 minutes to comply.
If you missed the story, that's understandable. It unfolded in the shadow of the administration's retreat from the Iran war. But in the long run, this may prove more consequential because the fight between Washington and Tropic is not really about one company. It is the first visible battle over who governs artificial intelligence and whether that governance will happen through rules and institutions or improvisation and raw power.
By now, it should be clear that AI is a general-purpose technology that will flow into everything human beings do, from business to government to personal life. And increasingly, it's clear that it may be as powerful as imagined. Senator Mark Warner recently recounted testimony from General Joshua Rad, who heads both the National Security Agency and Cyber Command that Anthropic's most advanced model, Mythos, was able to penetrate almost all of the government's classified systems, not in weeks or days, but in hours. The question now is not whether to regulate AI, but how.
The Trump administration's approach has been deeply confusing. Over two and a half weeks ago, President Trump signed an executive order establishing a voluntary review process for frontier AI models and giving agencies 60 days to design a framework for identifying systems that required heightened scrutiny. It pointed towards something sensible, structured oversight, advanced testing, clear thresholds. But before that process had even been created, the Pentagon had declared Anthropic a supply chain risk, and the Trump administration at one point directed federal departments to stop using its systems.
The administration's concerns are not frivolous. Anthropic appears to have made real mistakes. A Washington Post report suggested that it expanded access to Mythos beyond what officials believed had been approved and moved too slowly in responding to concerns about who was being allowed to use it. Intelligence agencies reportedly favored a tougher approach. But that is precisely why process matters. When a technology becomes this consequential, decisions cannot be made ad hoc by whatever faction wins a weekend bureaucratic battle which might then get reversed the next weekend.
In its 90-minute demand to Anthropic last week that effectively forced the company to take its new model offline, America was telling partners around the world, "If your economy becomes dependent on American AI infrastructure, Washington can arbitrarily and without warning or explanation use its on/off switch." Unfortunately, personalized, arbitrary, and unpredictable regulation seem to define this administration's approach to business. Rather than doing even industrial policy by setting out rules and criteria that all can play by, it simply jumps in to take a stake in Intel, forces Nvidia to pay it a special tax, takes a golden share in US Steel, often with unclear legal basis for its actions. On other cases, its actions seem tied to connections and investments made by the friends and family of the president. This is no way to run the world's most advanced capitalist country.
There is plainly bad blood between the Trump administration and Anthropic. In the initial fight between the company and the Pentagon, Pete Hex accused Anthropic of betrayal and duplicity. Senior administration officials have repeatedly criticized the company, saying it was woke and hired too many Democrats. President Trump has publicly mocked the company. AI is too important for this kind of score settling.
There's a better way forward, one rooted in an American invention that has worked extraordinarily well. Banks became crucial to the capitalist system, providing capital which was the lifeline of the American economy. But banking is prone to crises which have huge consequences. So the United States created institutions. The Federal Reserve remains one of America's greatest institutional inventions because it combines public authority with private expertise. It communicates constantly with markets while preserving independence. It conducts examinations, runs stress tests, establishes capital requirements, publishes guidance, and intervenes in graduated steps. Most importantly, the rules apply broadly and consistently.
This is roughly what advanced AI needs. We should create a Fed for AI, an independent institution staffed by experts in technology, national security, and business. It would require Frontier developers to provide pre-release access for evaluation. Establish transparent thresholds for dangerous capabilities, build a ladder of responses, warnings, remediation, conditional deployment, restrictions rather than abrupt bans, and make those rules apply equally to everyone. America should build such a system. Europe and Japan could create parallel institutions. Democracies must coordinate standards just as central banks do coordinate for financial stability.
This is not an argument for slowing innovation. America's banks have dominated the world because they are well regulated. Not despite that fact. But speed without institutions is not dynamism. It's dangerous volatility. AI will become the most important technology of our age. It deserves regulation that is intelligent, transparent, flexible, and consistent, not personal, punitive, or improvised. The challenge is not choosing between innovation and oversight. We've shown that we can build institutions capable of preserving both.
Also, a new announcement from Anthropic today is making news. Development of AI is moving faster than some had anticipated and Anthropic is finding that humans are needed less and less as AI is getting closer to effectively building more advanced versions of itself writing its own code. That's the warning from the Anthropic Institute, which today made some suggestions, saying, quote, "We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology." But they say if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe.
Anthropic goes on to explain this would require AI labs in multiple countries agreeing to stop under the same conditions or work under the same conditions. Jack Clark is an Anthropic co-founder and head of public benefit. He joins me now. Mr. Clark, you said that we are turning toward an quote AI system capable of fully autonomously designing and developing its own successor which is called recursive self-improvement and that it could quote come sooner than most institutions are prepared for. Let's start with what is recursive self-improvement exactly?
The original goal of the AI industry, the AI researchers, has been to build systems smart enough that they could become a a generally competent scientist and make scientific discoveries. One of which would be figuring out how to build better AI systems. I think it's highly likely now that we're on the the cusp of developing systems that are capable of just that and they may come along in in the next few years rather than decades, which is a lot sooner than I think people had been anticipating. The upside potential of this is huge as is the potential risk of this.
So let's just talk about the upside. What is the benefit of this? So today when we want to do good in the world in areas like biology or medicine or robotics, you have to take these systems that mostly exist as digital AI systems and adapt them into this complex domain in the real world. And you do that by pairing with scientists to do it. Now, an AI system capable of improving itself is also capable of going into domains like medicine, going into domains like biology, discovering what it needs to get smarter and more capable, and working with people to adapt itself to improve its performance there. So I think what we're seeing is the potential for a dramatic acceleration in science in the coming years as these AI systems gain this capability and become much more like creative co-scientists than tools that scientists use which would be a big change.
The the downside I mean for anybody who's seen any science fiction movie and I hate to be so sort of uh given my lack of science knowledge a lot of it comes from science fiction movies but in obviously in all the science fiction movies we give control to these machines and obviously we all know what happens and and the people who create them are the first ones who get killed and then they run them up in in human society what is the what what to the risk here? Yeah, we read the science fiction and watch science fiction here as well so it's not lost on us that this is how some of the stories start. And the risk here is what happens if you can't validate or verify or trust the behavior of these systems. It would be like if we dropped hundreds or thousands of new colleagues into your newsroom. It would take you a while to figure out if you can trust them, if they work in the way that you expect, if when you ask them to do things, they come back with something that you think is good and in line with your expectations. That's one of the challenges here. How do you maintain control over over fleets of scientists that are much much larger and much faster than ones you've had before?
So, Anthropic wants to see the industry as a whole come up with a way to essentially, in your words, slow or or have the capabilities of slowing or temporarily pausing um AI development to let society keep up with the advances. Is that right? And because what and what would that look like in practice? And how do you convince competitors to do that as well? Because obviously this is, you know, obviously it's been described as an arms race.
So our view is we've built amazingly powerful technology. We're going to keep building it. And in the coming years that technology is going to start to do a lot of major things in the world in domains like science. But when I look down at the the car we're driving, all I have is a gas pedal. I don't have a brake pedal. And surely at some point in the future, we might want that option. the option to say to ourselves, to other companies, to the world, what would it be like if we focus now on on g taking these scientific advances we've created and pushing them through to the world and take our foot off the gas of just accelerating the AI systems.
How do you start? Well, you start by saying that this is something that you might want. So, we've we've said this today because we think that it's important that companies are out here saying it would be nice to have that option. that allows us to then talk with other companies, talk with governments and say how would such an option come into existence. And look, we've done this before in the height of the Cold War under highly tense situations between rivalous countries. They found ways to stabilize aspects of the nuclear arms race. All of this has been done before in other domains and it may need to be something we do in the domain of AI.
Yeah, it it's incredibly fascinating. Uh Jack Clark, I really appreciate the conversation. Thank you. Thank you very.