Transcription
This summer, joins me for a season of AI and basic sciences, a special series focusing on important and fun discussions about basic sciences and artificial intelligence. [snorts] Tremendous resources are now putting into developing the latest large language models in the US and in China. However, enterprise AI, the complete system that helps an organization solve business problems, also enjoys great business potential. That's suggested by Dr. Kai-Fu Lee, founder and CEO of 01.AI. He wants to build an upgraded version of Palantir and he hopes his company can strive to become China's first profitable AI 2.0 company.
As a tech veteran, Kai-Fu has witnessed and actively participated in more than three decades of artificial intelligence research, development, and investment in both the US and China. Earlier with Apple, Microsoft, Google, and now with his own entrepreneurship. Now he says he is all in as an entrepreneur on enterprise AI, both in China and in other economies along the Belt and Road Initiative.
Now, you have been transforming your business from LLM to enterprise AI. You made that decision almost a year ago. More than a year ago. >> More than a year ago. >> About a year and a half ago. While we did a pretty good job on building our LLM, it was, in fact, top-ranked in China at the time, but it became very clear that our Chinese, um, uh, competitors have 10 times or even 100 times our resources in training the next model and the model after that. And the American companies have several hundred times, maybe a thousand times more resources. So it's just not a war that we can possibly win. Um, because it is about the GPU and about the people. Our people are good, uh, as good as anybody in China. Uh, maybe a little bit below the American levels, but still, uh, taking a good team that is not, you know, head and shoulders above all the other teams with, you know, 1% of the GPUs. Uh, you're basically fighting a battle with a sword when they have, um, machine guns. So we can't win. So we must find a way to pivot.
Was it difficult for you to realize that? >> Uh, the writing was on the wall. Uh, it became, I think the signal that made it very, very clear was when Elon Musk built his Colossus, announced the plan to build a Colossus cluster, and we saw the numbers. It was, uh, it was, um, mind-blowing. >> Astronomical. >> Yeah. At the time, it was 100,000 GPUs, and now there are much more. But at the time, a year and a half ago, 100,000 was unthinkable. You know, we had 2,000. So, we're just going to be so outnumbered.
And interestingly, despite having that many GPUs, Elon Musk is now withdrawing also from training large language models. Um, the other realization really is the market doesn't need a dozen large language models. Um, and, and that they're becoming increasingly similar in performance. One overtakes the other, then another one overtakes them. So actually, um, they're all roughly equal on a one-year average basis. Uh, and it's such a pain for customers to have to switch back and forth. So one of the services we provide to the customers, which they now appreciate, is, uh, they ask us, "What model do you use?" And we tell them, "Don't worry, we'll always use the best model for you." And because we don't build models, um, you can trust us to make that choice for you because we're professionals. But from a uniform large language model to enterprise AI, that you have to cater to everybody's needs. I mean, the total concept >> is different, and the business gene, shall I say, is different.
That transformation is also interesting, I would assume. >> Um, we've always had a B2B effort within 01.ai, except before it was overshadowed by the large language model team. So this team basically came to the forefront, and for them, it was very exciting, but it was a small team. It was maybe, uh, 15 people, right? And then we just built everything around that 15 people, >> and then we grew that by leaps and bounds while many of the people who used to train large language models decided to leave. Uh, we helped transition many of them to Alibaba, but a few stayed with us, and a few went to other places. But those people who stayed with us now become very unique, um, value-add because very few enterprise software companies have people who know how to train models, and we still sometimes need to do that.
What do you think is the latest development? You see China, US. I'm sure you answer that question many times, but things are always changing. China, US taking different roads. US is more reaching for the stars, so to speak. China is more AI plus making AI everywhere. Uh, so how do those two approaches, uh, seeing their results distinctively at this moment? Do you see both sides are looking at the other side and say, "Hmm, not bad, maybe I should do that too"? >> I don't, I don't think so. I think the US remains the breakthrough, uh, research leader, frontier models. >> And China has more of, um, um, study group mentality. So if the US, clawed open AI is like a genius kid who wants to win the Nobel Prize, the Chinese companies are more like a study group that happens to compete with each other as students too for the, see who will be number one, but they find it better to study together, uh, so that they can all improve. So the study group effort is, uh, also happens to be open source, while the US is closed source. So they both mirror the industries that developed in the US. Enterprises generally trust the cloud. So they can send their data to the virtual private cloud on which the model is being hosted, and then trust that the cloud provider would not violate the agreement not to leak their data. So that actually matches well, and it creates a huge revenue stream to Anthropic and OpenAI. Whereas Chinese companies generally want, um, on-premise deployment. And if you want your model on-premise, you've got to use the open-source one because you can't send your data anywhere. So that also creates a completely different approach in the ecosystem >> at this moment.
Related to this, uh, is whether people, businesses, institutions, they should tap into the LLMs and services from all sides so that they can have more of a comprehensive plan. >> So when one thing is off, they can have other options. That's, uh, another thing that people are discussing. Meanwhile, cost-wise, open source at this moment is always much lower cost. Uh, the Chinese models at this moment, much lower cost. So, uh, cost and benefit-wise, how do you see this, you know, the salad bowl companies, enterprises, they should have? >> Well, I think every company should understand it's different workloads. >> Because there are certain workloads that don't even need the best open-source model. Maybe a very average model will do, as long as it's really cheap because the workload is very simple. So I think one needs to match the workload, um, with the capability of the model, both to save money, um, and also to, um, yeah, just to waste not waste money. So I think that's something, um, a product like OpenRouter has done, and there will be other versions of that in the future. Um, the Chinese models are cheaper if used on the cloud, but if you want on-premise deployment, some of the models can really cost a lot of hardware, so one has to consider that as well.
>> So congratulations on 01.AI, your latest development, especially on enterprise AI, but this is very competitive. >> Yes, I think we're the first to build some of the enterprise products. Yeah. Now you are aiming at number one of the company, suggesting the number one of the company has to be aware of the transition of AI and be at the front line. Tell me more about that. >> Uh, yeah, a lot of CEOs think of AI transformation as getting more people to use chatbots or building some basic tools like customer service, legal document review. These are all useful, but I think they, the problem is they think of AI as a piece of software that I buy and use and get some productivity out of it. Um, and I pay per seat and let everybody use it. But actually, we see that AI is getting so much smarter every year. And if it's getting this smart, uh, isn't it insulting for such a brilliant genius to say, "Review some documents for me? >> Uh, look up something for me? Reserve a restaurant for me?" These are not the best use of these super-smart AIs. Maybe the early, uh, ChatGPT, perhaps. So when AI gets this smart, where you have to use them is to help the top executive, the CEO of the company, uh, think strategically, make the right plans, get the right insights, uh, see the risks, and know who your top performers are. Basically, help you do a better job managing the company by being your, um, ears to the ground, knows everything happening in the company, and summarizing the biggest risks and opportunities, and help you gain insight and make decisions. That's if you have a super-smart brain in a company that is the one thing, uh, that is the most valuable, and it would also bring about, uh, a true understanding of what AI is capable of to the CEO, who will then be able to know how to start the full company AI transformation.
>> But one could argue the CEO level, at least for now, is not the one best with technologies. One could argue they could be a little bit clumsy compared to their technicians. >> Well, that is a misunderstanding we have to overcome, right? Because we obviously know most CEOs don't use CRM, ERP, or any other of the, uh, enterprise software that they bought. But this one is different. It's created to be a very friendly interface where you ask it any question, and it either answers, gains insight, or gives you choices. For example, any CEO can certainly ask, um, of all the meetings that happen in the company, uh, in the past week, what are the three biggest debates that I need to be aware of? Um, of all of my employees, who are the most productive in using AI to write code? Among these people, which ones are retention risks? For the retention risks, what are the reasons they might want to leave, and how can I save them? >> Total transparency, and it seems that the middle-level managers are not necessary anymore. [laughter] >> Uh, well, I think it certainly, uh, takes from the very top to everything happening in the company. Um, but I think you, middle managers are still needed because all we're doing with this Boss AI, who is the name of our product, is to answer for the boss the most critical questions he or she needs to answer, not to solve every small, medium, large problem in the company. Now, over time, middle management will flatten, there's no doubt, but that's another trend in the industry, which is as AI does more of the work, you don't need as many layers to manage the AI. You just, and also you might need a different kind of middle management that knows how to manage AI, but that's another story.
>> At this moment, digitalization can still be a challenge for many companies. Um, one could argue your product works very well if there is total digitalization. So how do you see companies are doing that transformation? These are very pragmatic issues, but they really matter. >> Well, digital transformation, um, is necessary, but it is easier than before. To give you an example, uh, to answer some of the questions I asked before, uh, of all the meetings in my company, uh, what are the three most important debates that were had? In order to answer that question, all meetings have to be digitalized. So we do require that. But the equipment to set it up and to record the audio, turn it into text, and know who said what, is has now become possible because of multimodal, uh, large language model-based speech recognition, natural language understanding, um, and speaker recognition. So with these technologies, uh, it's very easy to do. If five years ago, you wanted to say, "I'm going to do digitalization of all of our meetings. I want every meeting to be turned into text," it'll be a huge headache and huge expense to do. So in my company, uh, 01.ai, we record every meeting.
>> But enterprise AI is a very competitive area because people see the cash it could eventually lead to. So how do you see the competitiveness of this market right now, Kai-Fu, as China is all in now in AI? >> Um, well, the enterprise market is a very interesting one because in China, it is actually a little more difficult because, um, most companies do not have a habit of paying for software on a subscription basis. So that makes software projects project-based, which means the revenue is lower quality and not recurrent. So that is a challenge I think China does have to overcome. Um, and that is also why the Chinese model companies are not getting the kind of revenue Anthropic and OpenAI are getting, because enterprises are paying for software which uses their tokens. And if there aren't as many Chinese enterprises paying for software, then the software can't afford those tokens, then it, it hurts the AI economy. So that's the first issue I have to point out. Uh, that said, there are a lot of low-hanging fruits still in China. So we look for certain digital areas where there is, in fact, a need and a willingness to pay, uh, perhaps in the legal area, um, the, um, content area, uh, the gaming area, uh, but those are not our, uh, primary place. Um, and the other work we do in China is working with the local governments to get their help to get our tools to their, um, uh, enterprises, to have a chance to educate them and to come up with the right business model to sell to them, despite the issues that exist.