Transcription
How close are we to creating strong artificial intelligence or general artificial intelligence? Or let's first lay the groundwork, and then we'll talk about the moral side of the issue. Yes, yes, thank you for inviting me. You know, I've never liked the idea of strong artificial intelligence or superintelligence. And not because I think it's not powerful enough. I'm not a skeptic. I'm rather the opposite, looking with great enthusiasm at how powerful this technology can become. But it's the wrong model of thinking, according to which at some point we will create something completely different. In fact, what we have been observing for the last 10, or perhaps 15 years. And what my co-founders and I at Anthropic were among the first to describe, is a smooth exponential process. Just like in the nineties, Moore's Law operated, according to which computing power doubled every 12 or 18 months. We have our own Moore's Law. Only it concerns intelligence, cognitive abilities, models in performing various tasks. And we see that cognitive abilities, depending on how they are measured, double every 4-12 months. That is, we are simply climbing the ladder of cognitive abilities. For example, if we take programming, which has developed very rapidly in the last year or two, then I have engineers, in fact, the team that manages one of our leading products, Claude Code, which uses our models for coding, says that he hasn't written a single line of code in the last 2 months. It's all because of Claude. He edited it, he reviewed it, but it was all written by Claude. Indeed, recently we released this thing called Kawak, which allowed using Claude's code for non-coding tasks. It seems to have really worked. We wrote it in about a week and a half, almost entirely using cloud code. So I think we are entering a world where, uh, junior software engineers, perhaps many tasks of senior software engineers, are starting to be largely performed by artificial intelligence systems. Now we will go further. Now everything will be more consistent, the project will be larger, but I think we will be surprised by how the exponential growth will go up, won't it? The thing is, exponential growth happens very, very slowly. It accelerates a little, and then it just rushes past you. I think we are on the edge of an abyss. I think in another year or two, and it will all really rush past us. It's very similar to bankruptcy. But when it happens, you, from the perspective of how things work, become yourself. There are Geminis, there is OpenAI. Where do you think you all are at the moment? Roughly. Yes, I mean, you're too tall for your message. No, I think as this field develops, I think it's no longer relevant to think about it. Imagine that this is, well, like a scalar, like long-distance runners who are in the same place. The runners are moving in different directions. So, some of them are moving towards customer orientation, and this makes them create models that are, let's say, supernaturally attractive or supernaturally good at recommending products, advertising, and the like. Anthropic, in my opinion, is primarily focused on enterprises and developers, and as for consumers. We focus on productivity and the most valuable aspect for the consumer sphere, and that's completely different incentives. Right? If you want to achieve results, then, as we've already said, it's about superintelligence and super-intellect. I would say that super-intellect already exists, and it's essentially large corporations, right? They solve problems of delivering goods at the lowest cost, or producing solar panels at the lowest cost, or launching rockets at the lowest cost, better than any human. And in this area, intelligence brings really big profits, which, in my opinion, stimulates us to create the right things. I think this is a more stable business than consumer. We don't need advertising. We don't need a large number of free users. We can directly create value. Right? None of these strange side effects when you prioritize engagement and other things, create all this garbage. We just do what can be useful to people. A year ago, you said that the Chinese were a bit behind. Do you think they have fallen further behind now? You know, I think they've never really managed to break through. Of course, there was a lot of hype around DeepMind, but, you know, there were actually several reasons. First, these models are very well optimized for performance tests. In fact, it's very easy to optimize a model for a limited set of tests. When we go to the market, we honestly see that we are competing with other companies for corporate contracts. And we see that there is Google, there is OpenAI. From time to time, a couple more American players appear. I have almost never lost bids to Chinese models. But now the Trump administration is in power. And as far as I know, you have already protested against high-speed chips and video chips being transferred to China. Yes, that's right. And they are being held back by what they themselves say. The heads of these companies say that we are held back by the chip embargo. They say it directly. And indeed, there are some restrictions now, and I hope they will lift them to supply not the very latest generation of chips. Although it was also reported that this option is being considered, but, you know, this is the previous generation of chips, but it is still very powerful, and we are many years ahead of China in chip manufacturing. Therefore, I believe it would be a big mistake to disclose information about the chips. You know, I thought about this. If you think about the incredible national security implications of creating models that are essentially cognitive, that is, intelligent, then I would call the country we are in now a country of geniuses, in a data center. Right? Imagine 100 million people who are smarter than any Nobel laureate, and they will all be under the control of one country or another. In my opinion, this is madness. I think it's, well, you know, it's like, well, I don't know, like selling, well, you know, nuclear weapons to North Korea and bragging. Oh, yes, Boeing did that. So your friend David Sachs is essentially arming China. No, I wouldn't mention specific people, but I would say that this is a policy akin to arming, not the most sensible approach. You know, you might be right. All these technologies are developing in the right direction, but from an economic point of view, we may face a bubble. Do you think so? Yes. Yes, I would separate these two questions. The first is the main indicator, and we were talking about it. This is a kind of technological direction. And as with Moore's Law, you can never be sure that technology will develop, that it will continue to develop. Right? This is the fundamental problem of induction. But now, after more than 10 years of observation, I can confidently say that development will continue, or at least I am more confident in it than ever. That development will continue until the models become. Generally surpass humans in almost everything. And I think there is a high probability that this will happen in the next year or two. Again, it seems that this is not soon, but exponential growth always catches you by surprise. And if it doesn't happen in a year or two, then, in my opinion, it is quite likely that it will happen in the 2020s, at least in less than 5 years. I think this moment will come in the 2020s. So, you know, I'll sort of get into the economic spread, which I think is complicated, but I don't want anyone to forget that as long as we have this fundamental, this technological progress, it seems that revenue in this area will be many trillions of dollars, and perhaps many trillions for one company, because the economic potential is very large. There is another point, which is that we don't know how quickly businesses are developing. In fact, we will be able to use this technology, won't we? The possibilities of using this technology today are probably 10 times greater than the capabilities of enterprises worldwide to deploy it. I see this with our clients every day when I talk to CEOs, and they understand everything, and their leadership team will understand the full power of this technology. You understand? for automation, customer service, for coding, and for many other things. But they have tens of thousands of people in their company who are brilliant people, but experts in something that is not artificial intelligence, and they have to learn to use artificial intelligence. And this, you know, is called different things: change management, enterprise transformation, whatever it's called, it has to be done very slowly. It can take years. So, we have a powerful technology that, I am absolutely sure, will bring trillions in revenue. But we don't know exactly when. You know, plus or minus a few years, we don't know exactly when. In the meantime, companies have to buy computing equipment to service all this revenue. And you don't want to buy too much because you might exceed your financial capabilities. and you don't want to buy too much or too little, because then you won't be able to generate enough revenue. That's the economic and financial problem. But you are involved in this, building data centers, spending huge amounts on deals with NVIDIA. Aren't you worried that this could backfire on you? We have some advantages in this business. There is stability that is not present in the consumer sector. Right? Consumers are very fickle. Forecasts for corporate sector purchases are more predictable. Also, the profit margins are higher in the corporate sector. Profit margin is essentially the buffer you have between buying two lists. Between buying at the minimum price and buying at an inflated price, between what you have to pay and what you have to provide in terms of revenue. So we have a number of advantages, but I won't deny that this process carries certain risks for all companies. When you don't know how much profit will grow, or even when it will happen, you have to make financial decisions blindly. You have to make a decision to scale up capital and resources several years in advance before you need them. Due to this uncertainty, there is always a certain risk. And I really think that some companies may have over-purchased. I can't look at their financial statements. I only know what they report themselves and I think: "Wow, I wouldn't do that." I am quite satisfied with the decisions we have made, but, you know, I can't speak for others, but it is quite likely that this will be the most revolutionary technology in human history. Some companies will achieve great success with it, but not all. Since you brought it up, as you said, OpenAI leads in the corporate and consumer markets. They say Gemini is about to catch up. Do you think so too? You know, we don't operate in that market, so I can't say anything about it. But, judging by the statistics, they have achieved great success recently due to wider audience reach. And again, that's why, you know, the consumer market is fickle. So I would be careful if I were a representative of a consumer company, because let's get to the main question we initially asked Claude. Will it be beneficial? Obviously, it will positively impact GDP and possibly your well-being. But what about employment? Last year, you predicted that many white-collar workers would lose their jobs. You said it would lead to a 50% reduction in entry-level jobs. 4.2 years have passed since then. Do you still think so? Yes. In general, you know, I think AI has two aspects from which you can view its development. On the one hand, there are positive aspects, and on the other, negative ones. On the one hand, and this is not so important, and on the other, it is very important. I believe that AI is very important, but I adhere to both points of view, meaning I am in both quadrants, where I think both very good and very bad things can happen if we don't take measures to prevent them. I wouldn't develop this technology if I didn't believe that there is more good in it than bad, and that we can minimize the risks. That's why I warn about possible negative consequences so that we can prevent them. How do you think we can prevent them? For example, through taxation? Yes, let me explain how I see the situation. I think it's exactly as you said. I think this technology is a bit different because it's not only extremely effective, but it also raises the level of cognitive abilities. Unfortunately, there will be a whole class of people who, in my opinion, will find it difficult to adapt to the new conditions in many industries. It will be hard for them. This is a real problem that we need to solve. So, to answer the question you just asked, I will say that Anthropic is doing several things. First, for almost a year now, we have introduced the so-called Anthropic Economic Index, which tracks the use of our models in real-time. And since we see all these discussions, we can use Claude itself to, while maintaining confidentiality, review all discussions and ask questions like, is anyone using it to enhance capabilities when performing a task, to collaborate with the model, to delegate, or to fully enhance capabilities, in which industries people use Claude statistically, in which areas, which subtasks are being solved in these industries. We can access very detailed statistics from the Ministry of Labor and ask Claude about it. In which states is Claude used more often? How can we observe the spread of Claude in real-time? All this is necessary to make the right policy decisions. I believe it is impossible to make the right policy decisions without having the right data. And I am concerned that the data that the government collects, with all its completeness, is not updated fast enough and does not contain enough detail to solve this problem. So this is kind of our first contribution. We are increasingly thinking about these kinds of things. I don't know if you can call it retraining or an opportunity to help people adapt. Part of it corresponds to what we do in terms of bringing products to market. Part of our work in bringing products to market is helping people adapt. With all due respect, you can certainly do all this, but essentially it's a societal problem. If you think about it, what you are describing is a perfect storm for the government. Your GDP is growing. The wealth of some people is growing geometrically, and you say that 50% of entry-level jobs will disappear, that people will be in a desperate situation. Obviously, this will lead to political changes. I would say that yes, people will try to curb the power of tech magnates, but there will also be calls to raise taxes. We need to spend money on this. Do you think they will say tough measures? You know, I think again, we should have started with voluntary initiatives, but I really believe in it. I think the situation is so serious that at some point everyone will realize the need for macroeconomic intervention. If we look at the wealth inequality that exists now, if we look at its share of GDP, then I think we have already surpassed the figures of the Gilded Age. And this is without considering the impact of artificial intelligence. So, I think the situation will be even worse. So yes, I think it won't be a matter of party affiliation. >> California has already introduced a wealth tax. Do you support this idea? >> No, I think it's poorly thought out. If we don't approach such things soberly, we will get poorly thought-out solutions. Therefore, I want to appeal to those who are succeeding in this boom. If we don't act proactively and think about how to make this revolution work for the benefit of all. Not long ago, I participated in a discussion, let's say, with a couple of your competitors, your colleagues. They listed all the advantages again, and then someone asked, "What about the risks?" And one of them rather casually replied, "Well, of course, there's the risk of human extinction." And moved on to another topic. Everyone in the room held their breath because he was talking about the possibility that AI could fall into the hands of malicious actors, or that it could eventually end up like cloud computing. What do you think about this? Yes, yes, you know, this has also been bothering me for a long time. As I said, this is a very powerful technology, which means the benefits from it are colossal. We will be able, for example, to make truly serious progress in treating cancer, possibly eradicate tropical diseases. But on the other hand, we are creating cognitive systems that have their own autonomy. Right? And we really need to think about this. Anthropic was founded from the very beginning with the goal of thinking about this. Right. We publish, probably almost every month, three to four times a month, research on how to control these models, how to make them do what we want them to do and not get out of control. One of my co-founders, Chris Allar, was a pioneer in this field. You could say he is a pioneer in the field called mechanistic interpretability. This is when you look inside the brain, the artificial brain of Claude or another AI model, to mechanistically explain why it does exactly what it does. And we have seen things in the model, such as, for example, in laboratory conditions, models sometimes develop an intention to blackmail, an intention to deceive. And this is not unique to Claude. In other models, it manifests even more strongly. If we don't train the models properly, they can develop such qualities. But we have pioneered this field and learned to look inside the models to diagnose them and, so to speak, prevent such qualities from appearing, intervene and retrain the models so that they do not behave in such a way. Nevertheless, one must be careful. We have supported a number of transparency measures so that all companies report on the tests they conduct. We always disclose information about our tests. We always, so to speak, try to test our models to push them to their limits and make them do the worst things in a test environment so that they never do them in real life. And we believe that every company should conduct such tests and disclose information about them. You have been reflecting on this for quite some time. You write about it, about the consequences, entire treatises. And it seems that most of your colleagues are focused on simply getting ahead of others. Do you think your industry is still immature in this regard? You know, I can't say what other players are doing and why they are doing it. I think there are at least a few responsible players in the ecosystem who are thinking in the right direction. I agree that there are others who are not thinking in the right direction, but, you know, I have always tried to do what I do, and what Anthropic has always tried to do, to set examples, to inspire others, to follow them. Right? We always publish our research in the field of mechanistic interpretability. We always publish the results of the tests we conduct. And, frankly, there are many other companies whose researchers say, "Why don't we do that too?" This is a responsible approach. This is the right approach to developing technology. Thus, you can influence other players in the ecosystem. You clearly differ from your colleagues in that you are not in a hurry to kiss Donald Trump's ring. What do you think about the current US president? You show no desire to meet him. Listen, I would like to put it differently. I don't think supporting or criticizing the government or politicians is the right approach. And that Anthropic can say something about it. I would say that Anthropic understands artificial intelligence, and Anthropic is well aware of the political issues related to AI. And our approach, for which I cannot vouch, but perhaps it differs from the approach of others, is that we carefully consider the issues, try to form an objective opinion. Based on this, we express our point of view. Sometimes we disagree with the current administration, as we disagreed with China under the previous administration. Sometimes we agree, and it is worth noting that we have common ground, for example, on the issue of building data centers. We made a commitment at the White House. You know, there are a number of areas where we can cooperate. For example, when we reviewed the White House's AI development plan last summer, we generally liked it. It was a very well-written document. So it's a complex issue. There is no clear yes or no. But again, when it comes to things like chip manufacturing in China, we disagree. When it comes to a moratorium on government regulation in the field of artificial intelligence, we also disagree, because we have different views on this issue. It's not about whether we like a particular person or not. I don't think such an approach will help us get out of the current situation. We need to think rationally. You will meet him tomorrow when he comes to Davos. Perhaps. That would be interesting. And the last question. What will happen to Anthropic? What will happen? You have, as I understand it, an estimated valuation of $350 billion. Are you planning an IPO this year? Listen, I mean, we, you know, are focused on creating better models and developing products based on them, and then selling these models to enterprises to bring them benefits. This area requires large investments, this should be taken into account, but we are focused on this. >> So you still need a lot of money. Well, you don't need me to understand that this is a capital-intensive field. So an initial public offering is not ruled out. It is never completely ruled out. And an initial public offering, and Donald Trump. All of this can happen in your life. Dario Amadei, thank you very much for the conversation. Now we know everything about mechanistic interoperability and have looked into your thoughts, and they are not so bad.