Transcription
I always remembered when I joined Google, Larry Page came and talked to us, and he said the ultimate search engine should be one where you ask a question and get a single correct answer. You've been at Apple, at Microsoft, and president of Google China. I love Google, but they're an engine that has been powered by advertising. How long is that going to last? Do you think that's going to survive? I think there needs to be a business model flip at some point, and Google will fail to do that, just as any innovator facing the innovator's dilemma. In your last book, you said something like the U.S. will lead in breakthrough innovations, but China is better in execution. What does that mean? The major technology breakthroughs were almost invariably invented by Americans. Now, when it comes to execution, it requires additional capabilities.
Before we get started, I want to share with you the incredible breakthroughs coming on the health span and longevity front. These technologies are enabling us to extend how long we live and how long we're healthy. The truth is a lot of the approaches are, in fact, cheap or even free. I want to share this with you: I just wrote a book called "Longevity Guidebook" that outlines what I've been doing to reverse my biological age, what I've been doing to increase my health, my strength, my energy, and I want to make this available to my community at cost. So, longevityguidebook.com, you can get information or check out the link below.
All right, let's jump into this episode. Hey Kaiu, good morning to you.
Hi Peter, good to be back.
Yeah, it's great to see you, my friend. We're on flip sides of the planet. I can't wait until we're having this podcast and we're in different parts of the solar system. That'll be fun, but we need faster-than-light travel. You know, I have the fondest memories of coming and visiting you in China, in your different locations, and I have to say, my takeaway from—I used to come to China every year; you would host a number of the Abundance 360 members I'd bring with me—super gracious. I remember my takeaways were that, number one, there was an incredible work ethic from Chinese entrepreneurs. I remember you describing that work ethic as 996. Is that still the saying there?
Yes, definitely.
Yeah, a good job was 9:00 a.m. to 9:00 p.m., six days a week. That was a good balance of life. The second thing I remember as a key takeaway was, at least this was, you know, I don't know, a decade ago, and it's been some time, but in the U.S., entrepreneurs see the marketplace as the U.S. and maybe Europe. In China, the entrepreneurs saw the marketplace as China and Europe and the U.S. It was a much more global view. I am curious if that's still the view in the entrepreneurial world in China today, because I've heard, and I'm seeing comments, where we're sort of like going into two parallel universes where products developed in China are staying in China, and products developed in the U.S. are staying in the U.S. How do you see that?
I'm curious.
Yeah, I think a lot of the B2B is becoming very much a parallel universe. It's hard to sell B2B, especially given export control and geopolitical issues, especially in the deep tech areas, which you and I care deeply about. B2C areas are much easier. You know, Americans use Shein, Temu, and TikTok, and of course, Chinese use a lot of American products: Mac, Apple, Windows, and so on. So that hasn't been as affected. I would also say that in pursuit of scaling law AGI, while the efforts are separate, the collaboration, or at least the sharing of ideas, is pretty strong in paper publishing, open source, of course, with the notable exception of OpenAI and now Google, who don't publish. But they don't do it for geopolitical reasons; they don't want the competitor to see.
That is fascinating. We'll get into that because you've taken a very much open-source focused mindset, and there have been many, many that do. I have to ask a question. So you've seen— you've been at Apple, at Microsoft, president of Google China. You've seen so much in innovations. I mean, you're managing what, like three billion in investments thereabouts? So you've seen it all. And of course, here are two excellent books which we've discussed on my stage ages before. I am curious about something, and I'd love your opinion if you're willing, which is: I love Google for many reasons—what they've done, their investments, their mindset of driving breakthroughs—but they're an engine that has been powered by advertising. And they've been able to reinvest that. But what happens now when AI is giving single solutions, and the ad-powered models? How long is that going to last? Do you think that's going to survive, or is there going to have to be a business model flip for Google?
I think there needs to be a business model flip at some point, and Google will fail to do that, just as any innovator facing the innovator's dilemma. Because Google is critically dependent on advertising revenue, and to do the flip would require going away, losing all the revenue coming in, going to an at-best break-even value proposition of a single-answer search engine, and then rebuilding up the new business model, whether it's subscription or advertising. And that's going to cause a roller coaster ride, mostly downwards, for the stock price, and that's not something that a publicly listed company can do. It's kind of sad to see because Google is clearly in the best position to reinvent.
Handcuffs.
Yeah, it's handcuffs. Your quarterly earnings reports are handcuffs. Your stockholders aren't going to let you sacrifice or take the risks. And that's why a lot of companies that should have jumped to the next generation of technology never made it.
Right. It's such a pity because Google clearly has one of the world's top two AI engines and by far the world's number one search engine. And now we're talking about merging two areas in which they're the best, yet they can't win because of this innovator's dilemma. It's unfortunate.
Yeah. I want to dive in during our conversation into what you're doing now. So for the better part of 30 years, you were one of the lead investors in technology in China, but you also invested around the world, but typically in Chinese markets. And you flipped over from being an investor now to being an entrepreneur, right? And building 01. What was the positive moment? I mean, because I am curious. I mean, you've seen so many entrepreneurs and so many deals. I mean, just what I wrote down here was, you know, you've been investing in NLP tech, enterprise AI, AI-driven financial solutions, autonomous vehicles, autonomous software—a lot. But there was a moment in which you said, "Okay, I need to go and build a company." Why?
Yeah, by the way, let's call it Z1. That AI—we were flexible before, but now that OpenAI has taken the 01 name, we'll let them have it. We'll just be 01A.
It is.
Yeah, it really means recreating the world with 01 using AI technologies. So, yeah, I was content doing investment in the early days of AI, in the days of deep learning, computer vision, convolutional neural networks. I was super excited because in my 40-year career in AI, I basically saw two AI winters, and even the non-winter days were not that shiny. So finally, I saw, "Wow, this AI is becoming mature." So I was very excited.
It's not a fad.
No, no. Right. And I was in the position of being a venture capitalist, so I figured the role I should play is invest because I'm, you know, older, hopefully in some ways wiser and experienced, and knows technology and knows business. So I invested in about 50 AI companies, mostly in China but some in the U.S., and they did well. We now have 12 AI unicorns. We soon have half a dozen IPOs just from the AI companies being the first investor, which is pretty rare. So I kind of got on the ride and enjoyed watching from the back seat the excitement that my entrepreneurs went through. So that was good. But then, generative AI came about. We all saw and understood generative AI, but we didn't see how big it would be until OpenAI showed us with ChatGPT. At that moment, I realized that I could invest in the area in China and elsewhere. I looked at a bunch of generative AI companies, but then I realized that to start one that late, to start a generative AI company after ChatGPT had taken over the world by storm, would really be very, very hard for any entrepreneur because you're behind by six or seven years. And if you don't already have a team or products or technologies, how could I fund these people? Because China did not have really a lot of generative AI companies—there was one or two at most. And I just thought, "Hey, if anyone could do it, maybe I could do it." It would still be a long shot, but given my years of experience and people network and understanding of the technology and business, let's give it a shot. It may be a long shot, but I feel that when I'm really, really old—I'm old now, but when I'm really, really old and look back, I would not want to look back and say, "Hey, I just had a coffee and I decided to invest." And even if I won with a great investment building China's OpenAI and I were an investor, I would still have regrets because how could I, my love of my life, not participate in it this time? And also, I saw that if I did it, it really could work. I would have a shot; others may have a shot too, but I thought I would have a better shot because I could pull a great team that had the right ideas. And also, I saw the world kind of dividing up into parallel universes and that someone needed to do a generative AI for China; otherwise, the Chinese businesses and people would fall way behind, and all the work that Xi Jinping did to bring China forward could be lost if the world had generative AI but China didn't. So I thought I would do it.
It's interesting, right? Because, well, here's the question, right? If OpenAI and all the other LLM systems had been equally available in China as they are in other parts of the world, would you still have done it?
A good chance I might not. I would then have to think about the likelihood of success.
Right, right. That becomes the main factor, not as much as helping the Chinese people in business to have a solution. Even if it's not as good as OpenAI, it might be 50/50 in that case. But OpenAI decided not to make it available to China, so that was tough. I mean, I think everyone would agree every country and every human is going to need to have access to this infrastructure called generative AI. It's going to be your consultant, your doctor, your educator, your everything. It would be like denying a country access to oxygen or electricity.
Yeah, that's why when we founded 01, our vision statement was to make AGI beneficial and accessible. That's very similar to OpenAI's vision: make generative AI beneficial to humans. But we added "accessible." We wanted to stress the point that we want everyone to access it, no matter where they live, what their nationality, their income level, etc.
It is quite interesting that you've gone the open-source route. Can you speak to that a little bit? I just had Ryan on the podcast, and we were talking about open source. I had the Mozilla Foundation CEO on the podcast talking about open source. Why aren't all companies going open source, and what was your motivation for open source?
Right, well, I think a smaller company, a newcomer, really needs the open-source community because having 100 people trying to compete with 1,000 people at Google and starting 10 years late is a losing proposition if you don't somehow work with the open-source community to help each other make progress. So that's just out of a practical consideration. Secondly, we saw a lot of good work in open source from universities, from Meta, from Microsoft, from Nvidia, and we couldn't start our company without these, especially Nvidia Megatron, Microsoft DeepSpeed. Without these, it would have taken us much longer to start the company. So we said, "Well, if we're going to take from the open-source community, we should rightfully give back every model we make except the most rarefied model." So we would keep closed source the very, very best model that we make; everything else would become open source. That is a way of giving back. I know some companies open source everything, but we can't do that. We do need a business model and some commercial advantage. Also, we decided the way we would do open source is through the Apache license. We would not be asking people to get our approval for commercialization, nor would we put a limit that if you started making too much money or have too many users, we have to sit down and come talk commercial terms. We want everyone to have what we have, just like we took from Nvidia and Microsoft, their open source. They didn't ask for anything back, and we thought we also should not.
So you're putting everything up on Hugging Face?
Yes, for people to access and GitHub.
Did you see the movie Oppenheimer? If you did, did you know that besides building the atomic bomb at Los Alamos National Labs, they spent billions on biodefense weapons, the ability to accurately detect viruses and microbes by reading their RNA? Well, a company called Viome exclusively licensed the technology from Los Alamos Labs to build a platform that can measure your microbiome and the RNA in your blood. Now, Viome has a product that I've personally used for years called Full Body Intelligence, which collects a few drops of your blood, spit, and stool and can tell you so much about your health. They've tested over 700,000 individuals and used their AI models to deliver members critical health guidance, like what foods you should eat, what foods you shouldn't eat, as well as your supplements and probiotics, your biological age, and other deep health insights. The results of the recommendations are nothing short of stellar. You know, as reported in the American Journal of Lifestyle Medicine, after just six months of following Viome's recommendations, members reported the following: a 36% reduction in depression, a 40% reduction in anxiety, a 30% reduction in diabetes, and a 48% reduction in IBS.
Listen, I've been using Viome for three years. I know that my oral and gut health is one of my highest priorities. Best of all, Viome is affordable, which is part of my mission to democratize health. If you want to join me on this journey, go to viome.com/peter. I've asked Naen Jane, a friend of mine who's the founder and CEO of Viome, to give my listeners a special discount. You'll find it at viome.com/peter.
I'd like you to provide me a few charts. I want to share one or two of these with the audience at this point. There's one in particular that talks about the impact of GDP of the PC era, the mobile era, and the AI era. If we can put that up, let's talk about what that means. How do you interpret this?
Yeah, I think, you know, if we look at the global GDP, it's interesting to note that the PC era brought about an uplift of the global GDP, then it kind of saturated. Then mobile brought another, then it kind of saturated. Now, there are many factors to the GDP. I don't claim PC and mobile were the only factors, but they were clearly major factors that greatly enhanced productivity and changed the way we worked. We, as humans, do more or less the same things for thousands of years: we work, we play, we communicate, we learn. But the way in which we do them changed from PC to mobile, and I would say with AI, it would be, in some sense, a similar change. It would be a new platform that, rather than infusing a computer on every desktop or allowing anywhere, anytime mobile access, we would make super intelligent AI in every app. We would have apps that are super intelligent that could do work for us, that could give us answers. I think that is clear that this is not only the third platform revolution, the third productivity revolution, but by far the largest one because of how much value it adds.
There was something you said that you wrote about in your last book that I thought was fascinating. It's an approximate quote, but you said something like the U.S. will lead in breakthrough innovations, but China is better in execution.
Yes, fascinated about that. Please elaborate. What does that mean for entrepreneurs here in the U.S. and entrepreneurs in China?
Yeah, I think, you know, we've seen this through the mobile revolution and through the early days of deep learning and computer vision AI revolution that the major technology breakthroughs were almost invariably invented by Americans. That's because of the great university system, research labs, and a culture that encourages and rewards risk-taking and innovation, and an amazing early-stage venture community that allows new ideas to be funded. And also the patent system—all of that basically started in the U.S., and no wonder that the U.S. is best at discovering new technologies in the phases where new ideas were coming out. Now, when it comes to execution, it requires additional capabilities. I think the breakthrough innovation is less important, but more important would be figuring out what to build and being focused on building it and asking no questions and executing and working incredibly hard, in particular asking really, really smart people to say, "Well, you're not writing papers; you're writing code to get this out there," and to view success as a success of a product or a business, not a success of a paper or an award.
So it's the notion that a lot of AI researchers today are more concerned about the citations they get on their paper versus making something that is generating revenue and users. I see that it was fascinating, of course, that OpenAI turned on ChatGPT and made a very successful first user product. But I've criticized, and many have criticized Google for not having an app, right? You have to do a lot of steps between something you're doing to get to a generative AI. You know, Gemini search. So you think that Chinese entrepreneurs are better at execution and better at creating something that's a beautiful user interface? Is that the primary?
Yeah, but that interface is not just the artistic beauty, but rather using all the principles, again invented in Silicon Valley: the lean startup, zero to one, that is building them, MVP, doing A/B tests, and tweaking. It's really the availability of the internet as an instrumentation that allows entrepreneurs to no longer have to be Steve Jobs. You don't have to know what the user is thinking; you just test it and tweak it. If you work hard around the clock and measure the right things, improve the right things, you will evolve to the right user interface. That's where hard work becomes the oil that makes the engine work in building a good app. It's not just brilliance and insight. Brilliance and insight would have favored the American entrepreneur.
I mean, there's another thing going on with a lot of U.S. AI companies, which I call the race to AGI. And I'd like to show a short video of a statement by Sam Altman. Let's pull that up. What do you think about this? Whether we burn $500 million a year or $5 billion or $50 billion a year, I don't care. I genuinely don't, as long as we can, I think, stay on a trajectory where eventually we create way more value for society than that, and as long as we can figure out a way to pay the bills. Like, we're making AGI; it's going to be expensive; it's totally worth it.
So what's your reaction to that? How do you think about that?
Well, I think we all aspire to build AGI. I know you do. I've wanted it for 40-plus years that I've been in AI. And we're very lucky to be at the point where scaling law appears to still be working, meaning that if you throw 10 times more computing at the AGI problem, it gets smarter. So it's tempting and logical to want to keep throwing 10 times more compute every one and a half years or so. Of course, where this runs into some issues is, you know, is it a good investment? Once you're putting $50 billion into it, are you sure there would not be diminishing returns? And also, are you too focused on the breakthrough AGI and not enough on the application ecosystem?
Yeah, is it a bunch of researchers geeking out versus people building businesses?
Yeah, you know, there's another chart here I want to bring up, and the question is, as I watch the cost of models, in particular inference models and such, plummeting in price, is it a race to the bottom? I mean, it's fascinating that the single most powerful technology in the world is effectively free.
So this is your chart, Kaiu. Tell me what this means for you.
Yeah, actually, I wouldn't quite draw that conclusion about effectively free; it's eventually free. So given a particular technology, let's say GPT-4 in this chart, it started—it was launched in May 2023 at $75 per million tokens, and today it's at only $440, and it's using a better version, GPT-4, which is smaller, faster, better, and much cheaper—roughly coming down 10 times a year. This is a good thing; this is the market leader reducing price, and it's reflecting the lower cost that they've accomplished because GPU costs have come down. It's reflecting better technologies because you can get better performance with a smaller model. Just as we had Moore's Law, I think the scaling law is a law because we do seem to see every year and a half or so it gets better, and also the cost comes down 10x per year.
So I would see a conclusion of, "Wow, this is going great." We just can sit around, and then all the things we want that are too expensive will become cheap. But I would also have a word of caution because we are basically in the stratosphere going at turbo speed. So one year is a really, really long time. Just think: two years ago, we had no idea any of this was happening. One year ago, we were still complaining ChatGPT was hallucinating and didn't know anything. That's recent, and all that's been changed. So this industry is moving in one year what mobile probably would have taken seven or ten years to do. So a year is a long time.
And I would also argue that even at $440, GPT-4 is way too expensive for applications. For example, let's take a look at, let's say, AI search, the example we talked about earlier. I think if you took GPT-4 and used it to build an AI search, you would end up basically paying something like 10 cents or more per search query, and Google only makes 1.6 cents of revenue per search period. So you'd be on a fast road to bankrupt due to the cost of GPT-4, and that's not even counting you have to build a search infrastructure. That's just the LLM costs. It's just way, way, way too high.
I think more precisely, it is around 10 cents. So to summarize, you find yourself in a situation where ChatGPT is not available to China, and the people of China need generative AI. You look around and say, "Who better to do this than me?" And you've got a huge amount of experience, so you jump in and create 01A. So what's the background there? One of the products you showed me was Bego, which is beautiful and fast and apparently cheap. So let's talk about the history real quick of 01A, and then let's jump into Bego.
Yeah, in 01A, we realized we were way behind OpenAI. We were perhaps seven years behind when we founded it, 17 months ago. It was only 17 months ago, and I didn't have a team of engineers. I had to use the first four or five months to hire people. But even with that, basically, the playbook that I took was from my own book, "AI Superpowers." We said we're not going to beat OpenAI at their own game. Can we build things very quickly? Sometimes the most challenging part of building something is proving an unknown idea to be feasible, which ChatGPT had done, which GPT-4 and now GPT-01 have done. With the leaders in research demonstrating that something is feasible, that is all we need to know because when someone builds a nuclear bomb or puts a man on the moon, for others to do it is much, much easier because empirically it had been demonstrated. So we were just saying, "Now we just have to be more diligent, read more papers, and work harder around the clock and leverage the strength that we have as Chinese entrepreneurs and engineers and just go 996 or longer if needed until we get to products that are competitive and efficient."
Right, because we probably can't win on accuracy, but can we make the equally accurate product much cheaper—cheaper to train, cheaper to inference, cheaper to train because we're poor; we don't have the $50 billion or $5 billion that Sam Altman talked about—and cheap to inference because we want apps to run lightning fast. That is what it would take for adoption because you want fast and low cost of inference. And in the last 17 months, we achieved all that.
So you just said something that's fascinating, which is access to compute. I think everybody imagines China has huge infinite resources, but it hasn't been the case in terms of GPUs. Does that scarcity of resources cause you to think differently and not be lazy or to be more innovative?
Yes. One is just the difficulty of acquiring GPUs given the U.S. restrictions, but also we only raised a small amount of money, so we couldn't afford, you know, 10,000 GPUs anyway. Basically, everything we've done, we did production runs on only 2,000 GPUs, which is a small fraction of what the U.S. companies are using. Elon Musk just put together 100,000 H100s, and OpenAI has even more. It's impressive, but we have basically, you know, less than 2% of their compute. But I am a deep believer in efficiency, the power of engineering, small teams working together, vertical integration. I also strongly believe that necessity is the mother of innovation. So I have a team; I told them all we got is 2,000 GPUs. We don't have 100,000. I don't need you to invent GPT-5. I want you to take a look at GPT-4, GPT-4, and can we match that in 12 months, in five months? And can, in the process of making it, all we have is 2,000 GPUs? You don't have a lot of compute. And when you make it, by the way, if you train it very efficiently, can we also have an inference that's very efficient, costing only a few cents to run it in apps?
So talk to me about your product, Bego. By the way, I asked you earlier how to pronounce it and where the name came from, and I think it's worth repeating so people remember it better.
Bego, and golden retriever, right? It's a dog name.
Yeah, yeah. I mean, we all know the name Beagle, B-E-A-G-L-E. It's a cute little dog, and golden retriever. But when the two of them make a little puppy, that puppy is called Bego, B-E-G-O.
Okay.
Bego is very good at hunting, and a golden retriever is good at retrieving, so it's an apt name for an AI search engine. But I should also point out that Bego is not a 01A product. It is a product that I did venture build, and it's actually an American company. It uses a model very similar to the model that 01A has built—super fast, super cheap—thereby thinking that AI search could be reinvented.
So do you want to talk about 01A, or do you want to jump into a little bit about Bego? Which do you prefer?
Well, yeah, let me start with 01A, then we go into Bego. So, yeah, 01A, I think, you know, this May we came up with a very good model called E-Large, and E-Large was a bit behind GPT-4, which came out one day later. We had the time where we ranked number seven, which is a great number seven model, number four company, just behind OpenAI, Google, and Anthropic. Something to be really proud of.
Let's pull up that chart one second. Go ahead.
So that's the May chart, but I want to talk about what just happened in October because over between May and October, lots of models emerged. E-Large was no longer competitive, but we had been working based on what I described as working super hard building to match GPT-4, and maybe even be faster. That was accomplished in October, and we kind of took revenge on the GPT-4 May version, which you can now see on this chart as just below us as number seven in the world. We just beat them by a little bit.
So this is a case in point where we saw GPT-4, we saw what it could do, and we knew it could be done. We said, "Let's go do it." Basically, with no hint on how it's done, we figured out how to do it ourselves. I'm sure the methods are different, but we did match their performance in just five months. Of course, in these past five months, other great models came out, including new versions of GPT-4, GR, and others. So we came out in October with a tiny model called E-Lightning because we wanted it to be lightning fast. This is a much smaller, much faster model, but it became the number six model in the world and number three company, and we also surpassed Anthropic this time.
How big was your team building this?
The pre-training team is basically three or four people. The post-training team was maybe 10 people. The infrastructure team, maybe another 10 people. So it's a 20 to 30-person project. It's pretty small. The thing we're most excited about and proud and unique about is that we trained this model—the pre-train—only cost a little over $3 million. This is 3% of what GPT-4 cost to train, and we actually beat GPT-4 in performance. The inference cost is very, very low; it's around 10 cents per million tokens.
And let's go to that. Let's go to that chart. There's a chart here that looks at inference cost over time, which is super impressive as well. So explain this chart here, please.
Yeah, so back in June, we were at $1.40 per million tokens cost, which was a lot lower than GPT-4 at the time, which was, I think, about $10 price. Then by September, we came up with a number of breakthroughs, including new ways of doing mixture of experts and better inference and ideas of KV cache management, etc. So we had a really big breakthrough, not in launching the E-Lightning because E-Lightning was 1/14th the cost of the previous E-Large model and at 10 cents per million tokens. GPT-4 had also come down in price, but it was $440.
So to a developer, just to give folks who are just listening and watching this, back in June, E-Lightning was $1.40 per million tokens. Today, it's at 10 cents per million tokens, and next June, it's expected to be 3 cents per million tokens. It's a 50-fold decrease compared to GPT-4, which was at, you know, $4.50 per million tokens. So, I mean, we are seeing this precipitated efficiency gains over time.
Yes, so, you know, another way to look at it is GPT-4 dropped 10x in one year; we actually succeeded in dropping 50x in the past year. So we feel we now have the most competitive, lowest-priced lightning engine. Our cost is 10 cents per million tokens; our price is only 14 cents per million tokens, so we're also not taking a big margin.
So if you look at the performance of GPT-4 and E-Lightning, their new version is a little better—not a lot, only a little better—but they're $440, and we're 14 cents. Incredible.
Is it all algorithmic gains?
There are a number of differences. I think we actually—I'm sure we use different algorithms because we don't know what they use. We came up with our own.
But I'm saying the improvements you're making over time, are they algorithmic gains?
Yeah, our performance gains going from E-Large to E-Lightning are using a new mixture of experts model and new ways of modeling, and also getting more high-quality, diverse data, and also having super-fast infrastructure so we can train multiple times to learn more and to do research. The efficiency gains were also mostly by the super-fast mixture of expert model, but also by some inference advancements in terms of KV cache memory management.
As an example, you know, the way we do our next-gen model design is not to go invent a bunch of new things and go make them fast, but from the get-go, they have to be fast. So we would first ask the question, "Where do we project in four months, which is our product cycle, the best chips might be, and how do we get those chips to inference really fast?" And, "Oh, these chips with a lot of HBM, which is high bandwidth memory, are coming out, and can we turn the inference problem from a compute problem to more of a memory-bound problem?" Then, "Should we rewrite our inference engine? How much RAM can we put on as a second layer memory? How much SSD can we put on? Can we construct a computer four months from now that is super fast?" I mean, it's made out of standard parts, but still, it has a lot of memory. Then we put the memory-bound inference engine on top. Then we asked the modeling team in four months, "What model can you build that fits perfectly into this box? Not too large, not too small, use up all the memory, but don't go too far and use power of two." So a lot of constraints for the researchers, which some companies might face reluctant researchers, but in our case, we are all building a product. We're one team in one direction, so we all took the order and then marched ahead, and out came a very accurate and super-fast model thanks to this vertical integration from model to inference engine down to the hardware and memory.
I love the old saying, you know, innovation comes from thinking in a smaller and smaller box when you put constraints on yourself.
Everybody, I want to take a short break from our episode to talk about a company that's very important to me and could actually save your life or the life of someone that you love. The company is called Fountain Life, and it's a company I started years ago with Tony Robbins and a group of very talented physicians. You know, most of us don't actually know what's going on inside our body. We're all optimists until that day when you have a pain in your side, you go to the physician or the emergency room, and they say, "Listen, I'm sorry to tell you this, but you have stage three or four going on." And you know it didn't start that morning; it probably was a problem that's been going on for some time. But because we never look, we don't find out.
So what we built at Fountain Life was the world's most advanced diagnostic centers. We have four across the U.S. today, and we're building 20 around the world. These centers give you a full body MRI, a brain vasculature, an AI-enabled coronary CT looking for soft plaque, a DEXA scan, a Grail blood cancer test, a full executive blood workup. It's the most advanced workup you'll ever receive—150 gigabytes of data that then go to our AIs and our physicians to find any disease at the very beginning when it's solvable. You're going to find out eventually; you might as well find out when you can take action. Fountain Life also has an entire side of therapeutics. We look around the world for the most advanced therapeutics that can add 10, 20 healthy years to your life, and we provide them to you at our centers.
So if this is of interest to you, please go and check it out. Go to fountainlife.com. When Tony and I wrote our New York Times bestseller, "Life Force," we had 30,000 people reach out to us for Fountain Life memberships. If you go to fountainlife.com/peter, we'll put you to the top of the list. Really, it's something that is, for me, one of the most important things I offer my entire family, the CEOs of my companies, my friends. It's a chance to really add decades onto our healthy lifespans. Go to fountainlife.com. It's one of the most important things I can offer to you as one of my listeners.
All right, let's go back to our episode. You know, one of the conversations over the last year is we're running out of data to really improve the models. What do you think about that? Do you believe that to be the case?
I believe it has a bit of a dampening effect on how much we can expect scaling law to continue, but I do think we have ways of getting more data, just not as easily as it used to be. Because the fact is that humans were smart to create language as something that could be passed on over millennia, and we have so—once we start doing generative AI, we took all the language data and put it on. Now, every year, we're generating more language data, but clearly way less than the total collection. So that is incrementally much, much slower. But on the other hand, we have video data, we have audio data, and also we're going to have embodied AI gathering spatial data. So those—and also AI—we have ways of creating synthetic data, which is not as good but better than not using it. So these are the ways I think we're trying to compensate for the fact that most textual data has been used.
If the outcome is, I think we will still get more data benefit, just not as much as we used to get.
Let's jump over to Bego. So it's a U.S. company, right? And why was it started in the U.S.? Is it something that was funded out of innovations? What's its mission? Talk to us about it.
Right. As I was building up 01A, I ran into a lot of brilliant American engineers and researchers. They want to stay in America, but they liked my vision. So I said, "Why don't I help you guys venture build a company?" So they built a company called Rhymes Technologies, and they build an excellent model, very similar in approach to the model in 01A. On that model, they added a lot of their unique multimodal and launched ARA, which is an open-source multimodal engine, which is one of the best in the world but only 3.5 billion parameters. Continuing the tradition that companies that I help build are very committed to open source, and on an advanced version of that area, they built an AI search engine. So I was pleasantly surprised when they showed it to me. In fact, I was blown away by how good it was already and also how really, really fast it was.
What's your hope with Bego? I mean, to come in and, through an app, become the dominant search player?
Yeah, I always remembered when I joined Google, Larry Page came and talked to us, and he said Google, in this current form, is not the ultimate form. The ultimate search engine should be one where you ask a question and get a single correct answer. That always kind of stuck with me. When I venture built the Rhymes and Bego team in the U.S., we talked about it, and we feel that the time has come. In building such an engine, we also consider, well, first, on a mobile phone, it's a very small screen, so you can't have all the tabs. Doing a research-oriented, multi-link search exploration is very, very awkward, and a single answer just makes so much sense. But of course, the first issue with a single answer is, what if it's not correct? What if there's hallucination or some errors? So we work very hard to maximize factuality, and Bego is actually better in factuality than a lot of the other AI engines measured by objective third-party queries. So I think I think those are really bringing us one step closer to Larry Page's dream.
I think right now the team just wants more people to try it, and they want really knowledgeable, caring, smart people to try it first and give them the most feedback. It's great that I can be on your Abundance program because those are the types of users your readers are.
Yeah, and how do you possibly compete against companies who've got billions of dollars in this field? Is it just that much better in implementation, that much better in alternative?
Yeah, it's a tough challenge. That's why very few companies go after the space. The fact that Perplexity gained some ground is an indication people want something refreshing. Also, I think we're confident about Bego's factuality, and it also has pictures inside the search, making it more engaging and entertaining. But also, I think just the search players, particularly Google and to some extent Bing, will be hesitant to replace their search engine with a one-answer engine because with one answer, people don't look at ads, and the ad revenue will permit.
Yeah, they will be. I have to ask the question that probably a lot of people are thinking: is this another TikTok where it's a Chinese-owned company, and it's a way for, you know, what, people's imaginations that it's just a way to get U.S. data into this?
Is this a U.S.-based company, a U.S.-owned company?
Yes, it's actually both U.S.-based and U.S.-owned. So it's not its employees are Americans, Singaporeans, Taiwanese. I myself am Taiwanese, so it's not a Chinese-only company—quite different from TikTok.
Yeah, I get that. I've been a fan of your work, Kaiu, for a decade now, and I've had the pleasure of calling you a friend. I had Elon Musk and Jeffrey Hinton and Ray Kurzweil, you know them all, on my stage at Abundance 360 last year, and there was a fascinating question that came up: the probability that AI will be the great invention versus the probability that it will destroy humanity, to put it very bluntly. I think Hinton—and congrats to him on his Nobel—and Elon said, "Yeah, 80% it's good, 20% we're screwed." Where do you come out on that? Do you have an opinion on this? And then how do we protect the downside in your mind?
Yeah, so, well, if we assume it's like a 10-year horizon, is that reasonable?
Yeah, I think all of it's going to play out in the next five to ten years. I think if we get through the next—my belief, I don't know if you agree with me—if we get through the next ten years, we're fine.
I totally agree. That's why I ask the question.
Okay, so in a 10-year horizon, I would say we have a 5% chance of a disaster caused by AI and a 35% chance of a disaster caused by humans using AI, and 60% we're good.
Okay, so you've written an entire book on this, but I'm going to ask you to provide some summarization. What do we do? How do we protect our downside? The upside is fantastic. Do you have any advice for parents, entrepreneurs, leaders here? How should they think about protecting our downside? What would you—if you're head of the world here, what do you do? What do you think?
Well, I think a lot of technological risks are best addressed by technologies. Like when electricity went out, the invention of circuit breakers. When the internet went out, the antivirus. So technologies are the best likely savior to technological problems. So I would encourage more computer scientists, AI people, to not just work on the biggest next big model or AI applications or AI inference or whatever, but some percentage of them, the ones who feel a responsibility and their conscience asking them questions, then they should jump into AI safety to find the various types of safeguards and guide rails that will protect us. I think to me that's the most important thing. Regulation comes second. I would actually feel general AI regulation to be in uncharted territory and potentially not constructive. I think it would be better to take existing laws—let's say laws about fraud, laws about other blackmail—and then apply the use of AI to achieve those things. Laws about slander, laws about theft. So we have lots of those laws. Those laws are effective and understood, so apply them to people who use AI to break those laws. Make sure the punishments are equally, if not more severe; that would create some deterrence. To start to regulate AI before it matures and while it's changing by governments that are slow-moving seems like a futile exercise.
Yeah, governments are linear or sublinear at best. You're going to be joining me on stage in one of my panels in Saudi Arabia in just ten days or so. Excited to see you there at the FII Summit.
Yeah, great.
One of the conversations we're going to have is around the potential dangers of ASI, artificial superintelligence. But before I go there, you know, I would argue that we passed the Turing test many years ago, and no one really noticed it. It's come and gone. Will we know when we get to AGI?
I don't know that there's a good definition of AGI, and I don't even know if there's a good definition of digital superintelligence. I mean, these are challenges when we talk about these words. Do you agree with that?
Yeah, I think AGI was created to mean that AI could do absolutely everything humans do, and that may not be the right definition because we can't yet project when AI will have love or even when AI will be viewed as having love. Those are still some distance away. But I think thinking generally that AGI just means something overwhelms us that does almost everything we do so much better, even the most challenging intellectual tasks, like inventing a new theorem or something in physics or chemistry. So if we kind of extrapolate that to be the ASI or AGI, then I think it's highly likely that it will arrive in the next five to ten years, and we do need to put in the safeguards.
I imagine we just saw a number of Nobel prizes related to AI. I have to imagine that in the very near future, every single breakthrough in physics, math, and chemistry is going to be enabled or driven or connected to some AI models doing the work.
Yes, absolutely.
Yeah, I met an economic professor on my recent trip to the Bay Area, and he said he already treats GPT-1 as a graduate student, one that's able to challenge him and find mistakes, and he will teach and guide the student, and the student learns. The two of them are great partners in inventing new things, so it's already happening.
And so I would add also economics to be perhaps another area to what you listed.
Do you say please and thank you to your AI when you're speaking to it?
I do say please. I don't say thank you. I'm not sure why.
It's interesting, right? I find myself saying please sometimes, thank you, but it is interesting to get how it's just a very small step away from being a part of every aspect of our lives. You know, people are worried deeply about jobs. You've made some predictions about the loss of white-collar jobs, and of course, all the multimodal AI systems that you're speaking about are being embodied in robots. There are a number of fantastic humanoid robot companies coming out of China. China needs robotics for its aging population, right? The one-child policy has significant implications for an aging population. So talk to me about your prediction about jobs, your advice. I know you wrote about this as well in your previous best-selling books, but if you just have a few, you know, how should people think about jobs, white-collar jobs, and then labor with humanoid robots coming? What should they tell their kids? What do they do for themselves? How do you think about that?
Well, I think the fact is that the white-collar jobs are going to be the first set most challenged by AI because just software can replace a lot of the routine and even non-routine work, and they will do so very rapidly in the next five years. That's, I think, now universally believed. When I wrote about it in my earlier books, it was met with a mixed reaction.
So everybody expected it was going to be blue-collar work leaving first.
Yes, yes, right, because it seems, you know, having intelligence in a white-collar job is harder to replace. But it turns out dexterity is harder to replace because that's not necessarily solved by the generative AI technologies. So blue-collar work, I think, going from factory to the type of caring work you talked about for the elderly is going to happen as the next wave. I'm among the more conservative on how fast that will happen because I think these technologies are very expensive. Not only do you need the LLM expense, but also these robots that Elon Musk has shown are way out of any consumer's price range. So they're kind of going to be a while before the kinks are worked out, before people accept them into their families and lives and offices, and the costs have to come down.
So I would project that—
You say that, but I'm an investor in Figure AI, Brett Adcock's company, and Figure and Tesla both are projecting around a $30,000 price tag. Let's say it's a $40,000 price tag. If you can lease that, right, and you lease it at $300 or $400 per month, having a 24/7 employee for $400 a month is pretty affordable, at least here.
Yeah, I can see your point. I can see your point. I'm still a little more cautious because, you know, especially used around the home, the "just clean my room" is one thing that I would predict in three years this robot cannot even begin to do because every room is different, every definition of clean is different, and every family home is different. But there are many other things, you know, like talking to the kids or doing more household repetitive work that can be done. $30,000 I think is probably a reasonable price point for middle-class America, but for China, India, and other countries, it's still way too expensive.
Real quick, I've been getting the most unusual compliments lately on my skin. The truth is I use a lotion every morning and every night religiously called One Skin. It was developed by four PhD women who determined a 10-amino acid sequence that is cytotoxic that kills senile cells in your skin, and this literally reverses the age of your skin. I think it's one of the most incredible products. I use it all the time. If you're interested, check out the show notes. I've asked my team to link to it below.
All right, let's get back to the episode. Kaiu, looking forward into 2025, 2026, would you mind sharing some predictions of what you might see in the AI world coming that would be surprising?
Sure. I think from the product side, we will see basically every category of mobile app incorporate AI, with many of them showing AI-first disruptive types of changes. So in other words, in about two years, every app we use will be replaced by another app or super upgraded by the same app. I think agents will be a major technology where we delegate what we want rather than just get an answer. I think multimodal will not only be dramatically better because a lot of the super smart people are working on it, and we're seeing text-to-video—something that would be a fairy tale in my youth is now starting to happen. I would also project that these great technology advancements will find real uses in applications, so it's not just demonstrating, "Hey, look what the AI drew for me in a video today," but "Wow, I created this marketing video for $10." Those are the kinds of advances I would definitely anticipate to see from a business products and some of the known technology side.
You know, we just—I don't know if you saw this—we just mapped the connectome of the drosophila. Did you see that? There was a—they were able to map 50 million synaptic connections, and it's a step away from a mammal, let's see, the mouse. I think we're going to probably see the connectome of a mouse done in the next year or two.
Where do you come out on the whole brain-computer interface world?
I'm still a little bit more, I would say, cautious about it. I think this is one of the areas where there's major disagreement on how fast this is moving and what dangers it might provide. I think we need to be cautious because it is intrusive to our bodies, and it's a kind of potential—a potentially slippery slope.
Right. I think people can all get on board with treatments using interfaces and get on board with non-intrusive kinds of BCI, but as we go deeper and deeper into reading our minds, creating star tissues, I just think we just have to make sure that people who are being experimented with are aware of what kind of risks they have and that the downside doesn't outweigh the upside.
Let's wrap up with a quick look at something I've heard you speak about, which is you were there at the PC revolution, the mobile phone revolution, and the AI revolution, and you've seen those progressions. I think you've modeled what the progression will be for AI. Can you give me that summary? Because I think it's super useful for entrepreneurs listening. If you're looking at starting a company in the AI world, there are a lot of lessons learned from the PC and the mobile phone world.
Yes, I think applications always follow a reasonable pattern of being replaced because when a new technology, a new form of content comes out, you, as users, have to first browse them, then you make the content, then you search and organize the content, then the content gets richer into multimedia, multimodal, then you can transact on the content, whether it's by payment, advertising, e-commerce, or online to offline, because these are the fundamental needs of people. The progression of apps that I talked about goes from fewer users to more users, a small amount of usage to more usage, simple usage to complex usage. So it's a really exciting iteration of better technology that enables the next step on the trajectory: more people use it, more money is made, more entrepreneurs, more funding, more GPUs, more products, more models. So the virtuous cycle goes on, and the most exciting thing is it took the PC ecosystem easily 30 years to play out; it took mobile maybe 15, but we're going to see AI play out in the next three years or so.
So if you jump in to start the company, this is the biggest roller coaster ride you can ever imagine.
What's your advice to the entrepreneur jumping in to start a company in the AI space? What should they do? What should they not do?
Right. Yeah, I use the roller coaster as a metaphor because I don't see it as a rocket ship purely upside. There are a lot of challenges and traps. I would be cautious to probably look at an app company because that's the biggest space with the most entrepreneurs, and the inference costs are coming down. But when you think about starting an AI app company, be cautious first about can you handle the inference costs because those are too expensive. Don't run out of money before the—because inference cost is coming down. So time your launches, time your product design according to the technology you need and when that technology will be low enough in inference costs.
Secondly, be careful of the modeling companies because we've seen companies like Jasper who built great apps, but then the model sucked all their know-how because they saw the data. So ensure that you don't do that.
The last advice is all the models are getting better. One day they'll be close to AGI. Does that mean my app will be eventually limited to a veneer and with very limited value? I don't necessarily think so because historically we've seen great platforms emerge, but other apps can often build a moat. The moat that TikTok, Instagram, and others have—their value was not taken away by the lower-level transaction layers or operating system layers or browser layers. So the key is when you build an app and gain some edge, and don't sit on your laurels, think about how to build a moat. That moat could be your brand, your user loyalty, user data, or social graph—things that we have seen—or maybe new things in the AI era.
Yeah, I like to think about it, and I'm curious if you agree. When I'm evaluating an AI company to invest in, you know, I'm looking at what unique data do they have and what customers do they have a very close relationship with, and everything else in the middle will get demonetized and replaced over time.
Yeah, I think your advice is great for B2B. I was thinking more B2C. The two are definitely in concert.
Yeah. What should people know about—let's turn to the last question, which is we live on one planet, and we've got sort of this bipolar element of China versus the U.S. and AI, and we have this split universe. It would always be better to have alignment and everybody working well together, but what advice do you have there? How should people think about this? I mean, because it's a complicated way above my pay grade, and I don't want to put you in a situation where you're talking about anything that you don't feel comfortable about, but I can't not have the conversation of, you know, I see a lot of people feeling like China is the enemy there or the U.S. is being monopolistic there. How do we navigate the next five to ten years, which are the most critical?
Yeah, there are some things that we're just not able to change; they are what they are. But I think each of us can make our own judgments and decide where we can reduce the impact of this unfortunate geopolitical situation. For example, you know, open source is one area where all the countries collaborate equally and generously. Academic collaborations continue on areas of collaborations not involved in the sensitive model or semiconductor can still go on. I think, you know, connectivity in the world, working people to people, business to business needs to go on. It has to be good globalism, has to be right. Differences between governments is kind of like, you know, when our parents have fights with other parents, we kids can still get along and do something interesting and fun, right?
Agreed. You know, I like to say we all have the same biology, so a breakthrough in medicine in China is the equivalent of a breakthrough in the Bronx. We all share 24 hours in a day and seven days in a week. That's something every single human has, and so anything that gains time efficiency in one place gains time efficiency in another.
Yeah, and we share the same planet.
Yeah, it's facing its own challenges, right?
Yes. Thank you for sharing time. Super excited about their performance I've seen in Bego and look forward to playing with it. Thank you for joining me on Moonshots to talk about your passion, your vision, and congrats on going from the guy behind the curtain to the guy in front of the company.
Thank you.
Thank you so much. Be well. See you soon, my friend.
Bye.
[Music]