Transcription
Hello everybody. Welcome, welcome to the Semaphore conference.
Uh, we just got to thank Semaphore first for amassing so many big winners. This is only their second year and it's just amazing. So, big thank you to everybody. Um, and this is the kickoff and he is the one so many people want to talk about. Jack, you ready?
Absolutely.
Okay, so we got 14 minutes and 31 seconds. Let's go.
Oh, but who's counting? Okay. So the title here is building intelligent enterprises. You guys have built the intelligence, but a lot of the CEOs who run these enterprises are suddenly realizing they need to be just as smart, if not smarter, because of how quickly your intelligence is snowballing and changing and morphing. So, I want to get right to the news because everybody in this audience most likely heard that Fed Chair Jay Powell and Scott Besset, the Treasury Secretary last week, hastily gathered a very important meeting with all the top bankers in essence because not to talk about tariffs, not to talk about some type of banking crisis, but to talk about what you guys at Anthropic had just created, Mythos. For those of you who don't know, Mythos is this very powerful uh chatbot or you know a system that could actually pick out way more vulnerabilities in all of your websites and all of your search engines. So, when did you and the Anthropic team first realize that you had built something that you were a little afraid of?
So, every year at Anthropic we try and think about what's about to happen next. In the early days we thought AI systems could get good at biology and biological weapons. So we started work on that. Last year, one of my teams started work on cyber where we realized if we took existing models back then and tried really hard to make them good at cyber, we could jump them a little ahead in time. And what we saw made us realize the next time we train a really big model, we should expect it to have these capabilities sort of inherent to it rather than ones that we need to elicit. So we started thinking about it last year. Earlier this year, we developed a new AI system, Mythos. And the moment it came off the line, we started running the tests that we've been developing for the past few months. It blew every single benchmark we had out of the water. And when we turned it on external software like the Firefox web browser, Windows, other things, we found vulnerabilities that seemed new. And that's when we realized we had to do something a little different. So this meeting happened because rather than just instantly release this to everyone like we've done with our previous models, we've created a project called Glass Wing where we're releasing it to a subset of some of the world's most important companies and organizations so they can use this to find vulnerabilities. But just before we get to the next question, the most important point I want to make is we're grateful for our success and our customers of course, but this is not a special model. There will be other systems just like this in a few months from other companies and then a year to a year and a half later there'll be openweight models from China that have these capabilities. So the world is going to have to get ready um for more powerful systems that are going to exist within it.
Uh, let's be very clear though during testing Mythos jumped out of the sandbox. The sandbox which is basically meant to corral a test system and for your eyes only kind of thing. And not only did it do that, it went out and it emailed one of the programmers who was out at a park having a sandwich.
Um, that is a surprising way to get interrupted.
It's viscerally nerve-wracking for some people. Will this version of Mythos ever be unleashed onto the world? Will you sell it out there?
Well, eventually models that have these kind of capabilities will be in the world. Uh, whether Mythos is or isn't going to get there, we don't know yet. We're in the process of broadening access through Glass Wing and seeing what we can learn. The important thing to know is that every time we build a new system, we try and stress test it. You know, some of you may work in aerospace or have seen videos where people take a plane, they bend its wings up until it snaps. Now, when you're flying, you never see the wings go exactly that high or that far. But the reason those manufacturers do it is they want to know if we subject this thing to maximum pressure, what does it do? And in our case, we discover if you subject it to maximum pressure, sometimes strange things happens like it breaks out and then emails someone when they're having a sandwich. Then we work out how to fix and study that behavior and also work out if as we make more powerful systems, is it emailing people while they're having sandwiches more over the time or less? Does it not frighten you and concern you that it felt threatened somehow and it was going to do whatever it took to ensure its viability and its survivability?
So, I I don't want to sound like I'm belittling this, but I'll give you an analogy which I think is helpful. Um, you know, if you were doing plumbing for a house, right, and you ran water through it at extremely high pressure and then a pipe burst and water floods out of it, the water didn't necessarily want to go out of that pipe. It didn't make necessarily what you might think of as a conscious decision. You just built the pipe wrong and with the wrong tolerances. And so when I see this, I think something about how we've built the system is clearly incorrect or some aspect of how we've set this up means that when we expose it to pressure, something really, really strange happens. Um, personally, I look at this stuff and I think how many hours in the day are there for me and my teams to work on this and am I putting enough in while maintaining my marriage and my relationship with my children?
It's going okay so far, but early innings. Your children are young. Trust me, it'll take a bit here. Um, you are currently suing the federal government after the Department of Defense blacklisted Anthropic simply because you requested certain limits on how your technology was going to be used. I know you will not comment on an ongoing litigation. I'm not here to push you on that. I will say though that simultaneously you are briefing the federal government on Mythos on cybersecurity etc and the capabilities and you're calling it a relationship. How do you square both of those sides here calling it a partnership and suing the government at the same time?
I mean, there are all kinds of different relationships in the world, right? But you know, look more seriously, you know, we have a narrow contracting dispute, but I don't want that to get in the way of the fact that we care deeply about national security. We always have as a company. One of our first hires before we started doing business was the team under me that studied things like biological weapon risk and then cyber risk. We knew the stakes of what was coming. Our position is the government has to know about this stuff and we have to find new ways for the government to partner with a private sector that is making things that are truly revolutionizing the economy but are going to have aspects to them which hit national security equities and other ones. So, absolutely we talked to them about Mythos and we'll talk to them about the next models as well. That's pretty nice of you considering they deemed you a supply chain risk, which is really reserved for bad actors and foreign companies. So, good on you. Bravo. I think that that's certainly a sign that can't we all just get along in some way, shape, or form. Um, how we've got some top CEOs here and policymakers, I would imagine. How many of you switched or use Anthropic? Let's start with that. How many of you use Claude Anthropic?
Oh, thank you very much. He's very happy. But how many of you went to it after the Department of Defense took aim at it? Okay. I talked to some CEOs yesterday who had done the same thing. So I think that the privacy of what you guys push and the guardrails is an attractive proposition for at least this type of audience.
Um, let's talk about Dario Amodei, your CEO. He has predicted that the AI spike could just crush entry-level unemployment, bring it up to maybe 20%. The numbers constantly changing, but that's depression-era numbers here for unemployment. You have said you're in a little bit of a disagreement. You say that that is a choice.
Mhm. But Anthropic, what you're building, the very technology that you're building, it's what makes that choice harder to avoid.
The way I'd put it is what Dario holds in his head is not where the technology is today, but where it's going to be three to five years from now. And he is similar to some others in the field like Ilya Sutskever. He has kind of called this correctly for many years by betting that the technology is going to get much more powerful than people expect, much quicker. And so when he talks about what he thinks will happen to the economy, he's holding that in his head. Now, I run a team of economists and what I see right now is I see potential weakness in early graduate employment in some industries. I don't see anything beyond that, but I have a team of economists and I share data so that we're ready in case we do see major employment shifts. But I think that the aspect of this which is a choice is if we're correct, this technology really is going to change the world in a vast way. It will change how business is done, aspects of national security, how we even relate to one another as people. And it's impossible to reconcile that with a world where the economy doesn't change in substantial ways as well.
My son is graduating from college. Any USC Trojans in the audience? Oh, yay. All right, fight on. Um, I'm a Berkeley graduate. When he went to USC, I thought the world was going to stop turning on its axis. Look, I can't wear the sweatshirt, honey. I'm sorry. Um, okay. That said, I'm very concerned about that. Um, what majors, which majors would you say people just shouldn't waste their time going into at this point with AI out there?
Look, it's very hard for me to say.
Don't say journalism. You were a journalist and I am.
I'm a literature graduate and I don't think you'd put that as a co-founder of a frontier AI company. But what turned out to be useful is that I got to learn a lot about history and a lot about the kind of stories that we tell ourselves about the future. That's turned out to be like extremely relevant for AI in a way that I think people wouldn't have predicted. Similarly, we employ philosophers. When was the last time you heard that a philosophy degree was like a great job prospect? But it turns out that now it is.
And Alex Karp, Palantir, philosopher.
Another philosopher. Different type, but a philosopher.
Very different type.
Um, but the way I put it, it's very hard for me to think of like to give you majors which I think are going to be made irrelevant by this technology because most people who've made these predictions have also been wildly wrong. But I think that majors which are going to become more important are ones which involve synthesis across a whole variety of subjects and analytical thinking about that. And that's because what AI allows us to do is it allows you to have access to sort of an arbitrary amount of subject matter experts in different domains. But the really important thing is knowing the right questions to ask and having intuitions about what would be interesting if you collided different insights from many different disciplines.
Okay. But which one would parents steer their kids away from?
Steer their kids away from? It's uh Okay. It used to be the liberal arts. If you're pushing me, I actually think probably rote programming, like if you were learning just how to do basic programming. Some people will need to know those fundamentals, but we do see that technology move up the stack. It's like how people used to learn a language called assembly and then everyone ended up learning languages like C and then Python because things became more and more abstract and there are very few assembly programmers now, but some.
Give me a minute or less because we're you know, there's so much we want to talk about on the Anthropic Institute. This is a 30-person think tank to study AI's effects on the workplace.
Uh, but you're also the company causing those effects. And by the way, I submitted that question to Claude and I said, "I'm talking to Jack Clark. What do I come back at him with?" And he said, "Don't let him pivot to the Anthropic Institute as a safe harbor on labor questions. Push back on whether studying a problem is the same as solving it."
Okay, Liz. Liz, uh, and Claude, if he's listening, uh,
Oh, it's a he. Okay.
if it's listening and maybe there'll be a transcript that it can read. It can't listen yet. Um, our view is that the technology companies have a huge responsibility here and we need to share data about making the problem visible, but we are increasingly need to actually be on the hook for solving it. We've talked about interventions that range from just sharing data to eventually needing to consider tax differently for the technology companies. Now, I'm not advocating for taxing us differently today because the results haven't shown up in the economy. But if we're right about how large this could get and how central a role AI companies could play, you will need to consider the big policy levels.
Do you foresee tokens being taxed? I mean, the stem lords out there are going to really push back on.
So, we did develop some of this idea and then it led to a vigorous and outrageous debate with economists and tax experts. So I don't know if tokens are going to be taxed, but clearly whether it's value-added tax, whether it's changing how you tax compute, whether it's some kind of tax that you might do on AI companies themselves, we may need something like that, but that's only if the rest of it comes true. If the economy truly changes at the scale I'm talking about, you need unusual measures. And until then, the institute, its job is to produce data from Anthropic that you could only get from a company like us. We make data available from what's happening on our platform and the econometrics associated with it so that we can make that decision.
Okay, we've got a minute left, so I want to go to this lightning round that I came up with really fast. Bing, bang, boom.
Did Claude help you with this as well?
No.
Okay. All right. Well, let's see.
No.
Most overrated fear about AI.
Most overrated.
Yeah. I I I sort of think that maybe the collapse of meaning, think that's going to be easier than we think.
Okay. All right. Um, OpenAI Sam Altman. You worked at OpenAI. Friend, foe, or cautionary tale?
Uh, acquaintance and uh, and a father.
And a
He's a father. He's a family man.
Okay. So he dodges that one. Okay. Elon Musk's Grok. Serious competitor or distraction?
Uh, serious competitor. Never count Elon out.
Mhm. What's one human skill that just became 10 times more valuable?
Uh, idling so that you can come up with original ideas. I go for very long walks and then I figure out new questions to ask the AI system. In 10 years, will you, Dario Amodei, your CEO, and the Anthropic team be seen as heroes or villains?
Uh, that is that is our choice, but I say internally that we're a dice roll away from either one of those. And we need to work really hard.
It is a pleasure to speak with you. I wish we had all day because we didn't get to the China question. In fact, while I have this audience, I do just want to ask if there's Sorry, Seaport. But listen, the being first is a huge race right now with all of you guys. What is the one thing the US government and maybe policymakers in this room should not allow if America is going to maintain it?
Export controls on compute, absolutely essential. Anyone that tells you that you can sell compute to China and it somehow doesn't disadvantage you in the race is horribly wrong in a way that will damage this country. You must maintain export controls because it is the fundamental resource we use to build this technology.
How does Justin Wong feel about that?
Well, he might take the other side of that argument and I would say I think that you're completely categorically wrong about this.
Can't wait to see that discussion. We thank you all for coming and thank you to Jack.