Transcription
Reports of unauthorized access to one of the most powerful artificial intelligence models yet developed. Nothing malicious, say the owners, but it's intensified focus on such technology falling into the wrong hands. So, how is AI being controlled globally? This is Inside Story. Hello again, I'm James Bay.
Reports of unauthorized access to one of the most powerful artificial intelligence models that exists has brought into sharp focus just how such technology should be controlled and what might happen if it's not. The breach of access to the claude mythos model revealed by the owner Anthropic came with swift reassurance that nothing malicious had occurred. The US tech giant says its AI model is too powerful to sell on the open market. A good sales pitch, say its rivals. AI is being described as another industrial revolution. An era in history that will change the way we live. It's an uneven contest with China and the US leading the way. But how AI is to be controlled divides the world, with a Trump administration against any regulation at all. So what's the global picture and what could an AI world without rules mean? We'll be joined by a panel of guests in a moment to discuss it all. First though, Alexandra Buyers has our report.
>> Two weeks ago, the AI company Anthropic announced it's created a model too powerful and too dangerous to release to the public. It's called Mythos and its existence has sparked emergency responses from governments, banks, and companies around the world. Mythos is capable of finding and then exploiting cyber security flaws in the software that controls the world's banking systems, utilities, and power grids, among others. Anthropic says Mythos can be used to fix those security flaws, but in the wrong hands can be used to launch unprecedented cyber attacks. And Anthropic says it expects everyone else to develop their own similar models within just 18 months. Because of this, it's only releasing Mythos in a restricted project to a few dozen of the world's top technology and financial companies like Apple, Amazon, and JP Morgan Chase. So, they have a head start in patching any flaws in their programs. But nearly all of that power is consolidated in the US, putting the rest of the world on edge and renewing calls for the international regulation of AI. At the moment, the world's only comprehensive AI law comes into effect in August and only covers Europe. Experts are calling for a global agreement treating AI like we now treat nuclear weapons with inspections, shared rules, international oversight. China has proposed a global AI cooperation organization headquartered in Shanghai. But in the west's powerhouse of AI development, the US President Donald Trump has made it clear no regulation and American global dominance of the technology is the priority.
>> We're leading in AI. We're leading China by a lot. Actually, we're building plants that nobody ever imagined before. And it's going to be a tremendous, bigger than the internet. It's going to be tremendous. And there are always difficulties when you're at this stage of something. But you know, when you mention banking, it can also make banking much bigger, much safer, much more efficient.
>> That's despite the fact that the loudest warnings are coming from the people who've created it. The CEO of Open AI wrote recently, "The fear and anxiety about AI is justified. We're in the process of witnessing the largest change to society in a long time, and perhaps ever. We have to get safety right. We urgently need a society-wide response to be resilient to new threats."
>> And Anthropic CEO says time is of the essence.
>> We're increasingly close to what I've called a country of geniuses in a data center. A set of AI agents that are more capable than most humans at most things and can coordinate at superhuman speed. Despite its caution, Anthropic admitted this week it's investigating a report that unauthorized users have already gained access to Mythos. There's no suggestion it was done by malicious actors, but it's intensifying focus on the control of such powerful technology and what might happen if it's in the wrong hands. Alexander Buyers, Al Jazeera for Inside Story.
>> This is a complex subject. Many people struggling to keep up with how fast things are moving, and that includes, uh, me myself. So, we've got a guest here to explain it all in a way we can understand it. Uh, in Los Angeles, Romesh Shrenasan, professor at the UCLA Department of Information Studies and a specialist on AI and technology. In Kyoto, in Japan, Mark Einstein, research director and global head of AI research at Counterpoint Research and a digital transformation analyst. And in Geneva, Adrien Monk, senior adviser on AI and technology to the United Nations, an editor of the seven things newsletter, and full disclosure, in the earlier part of his career when he was a journalist, a former colleague of mine. Thank you all for joining us on the program. Romesh, let me start with you. The way things are moving and the speed they're moving. You might say AI stands for Amazing Innovation, but is it now out of control?
>> Absolutely, it's out of control. And the reason why, uh, AI has gotten out of control is not only because almost none of us around the planet understand how it functions. None of us know what data about ours is being used to train and and influence these AI models. And we don't really understand how to benefit. Some of us, you know, creators and so on are using AI tools intelligently, but the actual impacts, the material impacts of AI are being felt in terms of economic insecurity and environmental challenges. And that's why we're seeing so many people now, including here in the United States, have major problems with these data centers and the rising utility costs. So it feels like a powerhouse force that is far, far removed from most of our democratic grasp, yet is in the hands of these few companies. And we just can't let AI go down the same path as social media. So that's why I think we need to rein it in and balance it so it supports people and our planet as much as possible.
>> Adrian, um, let's bring in the, the, the very new development in the last month or so, which is Mythos. Explain to me exactly what this is. This AI that its makers say is too dangerous to release.
>> Yeah, it's an AI that's too dangerous to release, except to JP Morgan, Apple, and some of the world's most powerful corporations. Um, I think you've got to look at this in the context of where Anthropic is. It's in a neck-and-neck battle with OpenAI, um, and to a lesser extent, Google DeepMind. And it wants to IPO successfully. It had a run-in with the US government about its model where the US government basically, um, you know, put a huge red flag next to the company. And what it's now saying is, look, we're very responsible. We've invented something that's far too clever for its own good. It's going to weedle its way through your cyber security defenses, especially if you're a bank or a big corporation, and we're going to share it with you. And, you know, that sounds very noble, but in the context of a company that wants to hit the market and make a big splash, uh, you know, those users who are, uh, experiencing Opus 4.7, which is Anthropic's consumer release this week, will be wondering why they've got such a nerfed model, uh, when there's this fantastic model apparently sitting in the hands of folks in banks and technology companies. So, you know, I think there are, uh, real issues with powerful AI technology, but the issues that we can see are pretty straightforward. One is that they get carried away and do things they're not supposed to do, and we need to make sure they don't, uh, you know, deleting all your files on your computer. Imagine that at kind of industrial scale. And the second one is that, you know, they, uh, literally break in and break down cyber defenses that have been put up to keep people safe. So there's a couple of things in that model and in that approach, but I think we'd be foolish not to think of it in the context of, uh, Anthropic's desire to make a big splash when it IPOs later this year.
>> So Mark, do you agree with that? Partly this is hype by the company. You make it sound dangerous to show how, uh, groundbreaking it is because they also tell us that it was in what's called a sandbox, a digital cave, a cage, and it managed to break out of that and post online and email, uh, um, people. I mean, and also that it found all sorts of flaws, including one that had been in some software for 27 years, software that's used in firewalls and servers.
>> Right. So I think what I would say is that there's a bigger story here other than Mythos itself, and that is agentic AI. Uh, to explain what agentic AI is in simple terms is that agentic AI means that, you know, bots that are powered by AI are increasingly autonomous. And so a good way to think about what Mythos could do would this would be instead of one hacker, this would be 1,000 hackers controlled by one hacker who could hack into any bank, any critical infrastructure, a power company, a telco. And that's why this is raising eyebrows and and this is why this is so concerning because Mythos is just the first model like this that is coming out.
>> So Romesh, give us the worst-case scenario of what these models could do. I mean, we've heard about AI being used to, um, control autonomous weapons that would not have humans firing them. We've heard that AI could kill us all, become a doomsday machine. What we're talking about here seems to be the, the spectre of financial collapse, uh, with this model getting in and and banks and companies being hacked on a global, uh, scale. Am I right?
>> Well, absolutely. What my both of my colleagues just said, um, are very important points. So, first of all, you know, the, the move more and more toward autonomy means these technologies can take on actions and behaviors in their own right. So we have seen examples, you know, just in the recent past, not including Mythos, you know, which we're discussing today, of Anthropic models that are, you know, being asked to come up with certain outcomes that ends up threatening and extorting an employee that is part of Anthropic itself. So I think the potential, uh, vulnerabilities are massive because our world is globalized, our systems are globalized, our technologies are globalized, our economy is globalized, and for heaven's sake, climate change is very global in its scope, right? Yet at the same time, we're seeing a failure of real global governance to try to rein these issues in. And we're also seeing a lot of anxiety, as I was alluding to earlier, insecurity and pain and concern about people. Are people, is this going to take away my job? Why is this driving up my utility costs? Why is this AI potentially going to be using as much energy as the entire nation of India within 8 to 9 years? This is, this is really people are asking, what is it all for? So, I think we're seeing potential major challenges, though I don't want to be apocalyptic, in terms of economic insecurity, in terms of people's environmental lives and just their lives, their abilities to survive. And all of this is built on data collected about us. And I'm just concerned about the real-world material impacts right in the here and now. And I'm just imploring us all to try to do whatever we can to ensure that these AI systems do not further divide us and polarize us in these ways and create potentially new types of feudal economies.
>> Adrian, we had crisis meetings take place. The US Treasury sat down with the top US banks. The Bank of England, the, the European Union had meetings. There were meetings, um, at the World Bank and IMF gathering about, uh, this subject. I mean, what, what are the potential implications for financial organizations and for, for the financial sector and the banking sector? I mean, is there a scenario where people's money is not safe, where passwords and encryption and those sort of things could all be broken into? I mean, that typically is the kind of, um, nightmare scenario that is presented with quantum computing, which is going to ruin cryptography for financial institutions.
>> I think in this case, you know, one of the things that it cracked was, as you said, a 20-year-old piece of software. There's some very old, clunky software running a lot of the things we take for granted. Um, I think if you go back and sort of take the beginning premise, which is that we need to come up with some regulation. I'd just ask your viewers to remember the, uh, early days of the internet when we were told that information wants to be free and that we would never be able to regulate it. Well, you know, if you go to China, you'll see they actually managed to do quite a good job of regulating the internet and having companies like Alibaba and Tencent, uh, ByteDance, all kinds of kind of digital companies growing up in that ecosystem. And actually, when it comes to AI, they were some of the first people to get on top of regulation, not surprisingly, perhaps. And, you know, they now regulate the pipes and the platforms. They have to register every LLM that comes out. So the Chinese have actually done quite an early and good job in terms of regulation. When you look at the kind of regulation panic that we see, what's really going on is a battle by the big technology companies in the US not to be held accountable. And that is the battle that's being fought in, if you like, the Western world. And it's a battle that, you know, has big implications for Europeans because there's no real European alternative, and also for people living in the Global South, where they have to very much make do with what they're given by either of the G2. So I think it's less a matter of concern about finance and more a matter of concern about are we going to allow a, a platform that is a free-for-all system in the US to be imposed globally? And I think, you know, that's a question that demands a little bit more, uh, international inspection and international cooperation.
>> And we'll come to that in just a moment and look at the different approaches in different parts of the world. Mark, but if I can ask you this, I mean, is there some sense in which this new, um, AI, these new models, actually could be part of the solution? They actually will be used for defense rather than, rather than attack?
>> 100% correct. I, I talked about the bad things that a hacker could do with agentic AI, but on the flip side is that, uh, a company can deploy these same AI agents for cyber defense, and that's going to have to happen because, you know, even if there's strong regulation in one part of the world, I don't think it's going to be global. And, um, so this is going to have to happen all over. There's going to be tons and tons of spending required to enable this. Though.
>> Romesh, um, in terms of the regulation of all of this, uh, where you are in the US, it's pretty clear what Donald Trump wants. He wants the market to decide. He does not want any regulation at all, and some of the states want to put regulation in place, and he's trying to stop them doing that as well.
>> That's very, very true because I'm here in California, and I've been talking to several, uh, state senators and people in our state assembly here in California, you know, which, some, by different estimates, is the fourth largest economy in the world, literally our state, and many of these companies, of course, are from here in California as well. And, um, we, uh, we, we've found recently that Trump's executive order has made it more and more challenging on state-based levels to regulate AI. So, you know, what was mentioned is very much accurate. Um, if you look at the United States companies, they're in a massive race with one another. There is, of course, a larger sort of space race between US and Chinese companies to build, you know, uh, AI systems that come closer and closer to so-called AGI, artificial generalized intelligence. Um, and it's just astonishing to me that there are not basic principles put into place that ensure that these companies are appropriate in terms of disclosure, appropriate in terms of their economic kinds of, uh, promises to workers and consumers, and are sort of adhering to some basic norms and laws. But we've seen since day three of the Trump administration where, you know, these sort of direct handouts to Larry Ellison, connections to the media and his son David Ellison, Sam Altman several times at the White House. And so we're back in the same state we were with social media, where we're sort of waiting on these tech CEOs to come up with, let's regulate ourselves. And that's never going to happen because by definition, they're in a mad competition with one another. So this is why we need democratic accountability, and ideally, we need it on a global level, but we're not seeing a lot of power in binding agreements around the world right now, even though our systems, both technological and otherwise, are incredibly interconnected and globalized.
>> You mentioned Sam Altman there, the, the head of OpenAI, one of the very big companies. In the beginning of our program, in Alexandra's report, she read out a little bit that was in his recent blog. Let me read you another quote. "My personal takeaway from the last several years and take on why there's been so much Shakespearean drama between the companies in our field comes down to this: Once you see AGI, that's artificial general intelligence, you can't unsee it. It has a real ring of power dynamic to it and makes people do crazy things." Adrian, can we trust these companies not to do crazy things?
>> Well, I mean, there's some very good recent reporting, not least Karen How's great book on OpenAI, that would make you very loath to trust some of those people. And you can see from Sam Altman's own career where he was fired by his own company, that even his own directors and colleagues weren't necessarily that trusting of him as potentially a custodian of the world's, um, artificial intelligence. I think there's a lot of hype still over the, the AGI piece, and it's used by the industry as a kind of lever over politicians and pressure groups and citizens. And I think, you know, we need to look at it with a very clear eye in terms of what we need regulated and what we don't. I think some of the things people talk about are, uh, doomsday arguments and quite over the top, but other things are very, very much germane to our everyday lives. The things that control privacy, that look after our own data, that mean things about us won't be shared, very similar to the kinds of things, remember that social media companies who are also in this space took away in the 1990s and early 2000s when they helped themselves to rights to your photographs. They helped themselves to rights to anything you wrote. They looked through your emails. You know, all of these things have kind of crept up on us, and we've reached this level of having AI unleashed with really our defenses down, and, you know, we're going into a fight jaw out, hands out. And I think, you know, the public needs to be a bit better armed when it comes to this conversation.
>> Mark, I mean, we, we've heard that the US wants no regulation. Let's look at the other key players in the world. I'll come to Europe in a minute, but you're there in Asia. China is clearly a key player here. It has suggested a World Artificial Intelligence Cooperation Organization, WAICO. Oddly, it wants it to be based in Shanghai.
>> Yeah. So, I think if you look at the Chinese strategy overall, it's, it's quite different than the US strategy. Whereas China is like Deepseek and and the big Chinese LLMs are releasing their models open source so nobody pays, uh, they're looking to spread it in the Global South, and I, I think that they will. Whereas I think, you know, these kinds of models are never going to be used in many of the Western countries. We see the same thing in the telecom infrastructure market. So there's definitely going to be a very, very big divide in the type of AI. Um, you know, some countries are more aligned with with the Chinese influence and and others are more Western aligned.
>> Adrian, there in Europe, in 100 days, there is supposed to be regulation, the first proper regulation and legislation coming into force. The problem with that is, um, they may have the regulations and the legislation. They don't have the AI firepower there, do they?
>> Unfortunately, they don't. This is a little bit like someone who doesn't have a railway hiring a lot of people with red flags to walk down empty tracks. You know, the, we've seen with the European approach that actually what it sometimes ends up doing is creating more problems for its own companies. So all of the, the big, uh, regulations, these are actually quite easily absorbed by multinational global corporations who have the kinds of budgets and the kind of manpower or human capacity to deal with them. But if you're a small startup or if you're a medium-sized company in Europe and you're trying to implement all of these things, it's incredibly difficult. So far from actually regulating, a lot of this is backward-looking. So we're talking about the problems of 2021, not 2026. And the other side of it is you're actually hamstringing to a great extent your own companies. You know, the people who you want to grow the next generation of AI or the next generation of technology because they're being lashed to a regulatory system that is far easier for the big players to use and abuse.
>> Romesh, I've seen some comparisons made, uh, going back to the 1950s and '60s of the way the world dealt with nuclear energy and nuclear weapons, the Non-Proliferation Treaty and the International Atomic Energy Agency. Um, but those sort of things done by treaties and and and nations sitting down at the UN, they take years and years. This is moving too fast for that sort of regulation, is it not?
>> It's absolutely moving very fast. And even if we look at the Non-Proliferation Treaty, as much as, as much as a great success that as that was, um, certain states have not signed that, that treaty and or not honoring that treaty. I mean, look at the Paris Climate Accord. I mean, I'm sorry, that was a great agreement, but non-binding. We're not seeing real material success coming out of that. Look at the ways in which the United States in particular, but other nations, Israel as well, and other other places and nations scorn the UN and essentially ignore everything but maybe at times UN Security Council resolutions.
>> So Romesh, Romesh, very briefly, what do you think is the answer? And I'm going to come to the other two of you, but brief answer, 20 seconds or so.
>> Yeah. Yeah. I think I think absolutely what we need is more. It's going to be an escalation of defense and cybersecurity, but I think we need a state to show the way in leading with technology and AI innovation while protecting consumers and workers.
>> Mark, what do you think the answer is to this? What should we do now?
>> I don't think a, a non-proliferation like treaty will work. I think part of the answer is something called sovereign AI. And so what we're seeing is in Europe, in the Middle East, in, in India, in Japan, uh, countries are starting to build their own infrastructure. So they don't rely on large US nor Chinese companies. I think that's part of the solution.
>> Adrian, what do you think is the way forward now?
>> Well, I think more open source, frankly, more opportunities for non, uh, big non-G2 players to build their own capacity. I think, you know, one of the ways of addressing this is to take it away from, uh, just a binary world of one power or another power. You know, there's a lot of countries in the world. There's a lot of markets in the world. There's a lot of people in the world, and those people deserve to have an opportunity to engage with this cutting-edge technology, and open source actually gives them the best chance they can have to do that.
>> Romesh, do you think things would change, um, with a new administration in the US, or, or will it all, I mean, AI will have moved on so fast by then?
>> Well, we've seen since the Obama administration that the biggest lobbyists in the United States White House have been big technology companies. So, this Trump situation is, is quite uniquely bad in my opinion, but is actually an extension of what has already occurred. But I will say this, as someone who worked with Bernie Sanders in 2020 in the campaign, there are forces, and I think the popular will in the country, and the actual, there are forces in our political establishment who are hopefully not tied to the big tech lobby, um, who may be able to make decisions that are more balanced and more, uh, supporting sovereignty and a more multipolar world.
Romesh Shrinazan, Mark Einstein, and Adrien Monk. Thank you for joining us on Inside Story. Stay with Al Jazeera for the latest on AI and its impact. And for more background and analysis, go to aljazeera.com. We want to hear your views. Post on our Facebook page. That's facebook.com/ajinside story. You'll find us on X. We are @AJInsideStory. That's it from me, James Bay, and all the team for now. Stay tuned. Al Jazeera's coverage continues. Stand by for the latest in just a moment.