Transcription
Today's models are just giving you the output. Tomorrow's models will start with an output, reason, elicit feedback from the world, go back, improve the reasoning. That is the beginning of a real reasoning era.
The biggest beneficiaries of the commoditization of foundation models are the application layer companies ready to go. [Music]
Arand, I'm so excited for this. I've been looking forward to this one. So first off, thank you so much for joining me today.
Thank you for having me, Harry. I've watched a lot of your episodes, so looking forward to it.
That is very, very kind of you, my friend. Listen, I want to start with a little bit on you. How did you first fall in love with AI and realize that actually this was what you wanted to do and spend the majority of your career on?
It was not more like an accident. Um, I was just yet another electrical engineering or computer science undergrad doing my courses and doing some interesting projects along the side. There was a point when one of my um friends and undergrad told me, "Hey, there's this uh contest um where you could win win some price if if you came first." And I think I was like, you know, kind of like a need of money because I wasn't sure I was going to get an internship, so I I tried the contest out. And um, it was a machine learning contest, but I didn't even know what machine learning was. Um, all I knew from that guy was that, "Hey, you're going to be given some data and you can uh use some of the patterns in the data and use it to make predictions on you know, held out data that you don't have access to. The server will have it. You submit your algorithm and it'll score against what is correct and what you predict, and whoever wins the most number correct predictions wins the contest and you get the price." That that's the extent to which I I was told. And I go and um check out this Library called scikit-learn. It's a very popular machine learning library, and I have literally no idea what any of these words mean, like decision trees and random forests, like none of these things made any sense to me. Um, and so like I I just like literally just did what an AI would do: Brute Force random search, but as a human did all that, and uh we won the contest. I won the contest, and then that gave me a lot of confidence. Okay, uh I beat people who actually knew machine learning in it, and that gave me a lot of confidence that like this is something I could be pretty good at.
I remember Sam Altman once telling me—I asked him this question like um two three years ago—"Hey, like how do you identify something where you you're naturally good at?" And he said, "Whatever comes easy to you but seems hard to other people, like that's a good heuristic to identify things that you could be like mu plus 2 Sigma at compared to the rest." So I felt like, okay, this machine learning is a good good U thing. It was not called AI. Um, so I got into it, and I did a I did all the you know, courses, pattern and pattern recognition, machine learning, the book written by Christopher Bishop. I bought it secondhand in India uh for like you know, something like $23 or something, and and and started reading it, and I really enjoyed it. Like it was pretty mathematical but also intuitive at the same time, the kind of the sweet spot I really enjoy. And that got me access to a professor who is uh Rich Suon and Andy Ng's student, uh and he was teaching at my undergrad Institute, and uh I told him to advise me, give me a project, and he was um a reinforcement learning guy. So reinforcement learning is when I actually got into AI um because we'd written AI for like Checkers or like Tic-Tac-Toe or something like that, like you know, or or even when you play chess as a kid, when you play against a computer, you you always ask the question, "How does computer play? Like what is it like?" And then they'll call, "Oh yeah, it's like an AI. You don't worry about it." So they use AI loosely there, uh but the real definition of AI, what it is like, "Oh, it's an agent; it's it's it's an environment; it receives a reward signal; it optimizes for an objective"—all that framework mathematically uh made sense to me once I studied RL. And then he told me at the end of the class, "Hey, I have a friend of mine uh from UK uh his name is David Silver. You know, we used to know each other from PhD days, and and his startup just got bought by Google uh for like half a billion dollars because they wrote this paper that learned to play Atari games just from the screen pixels. Um, and um um they've open sourced the code. Why don't you take it and now figure out how to play every like like uh all the games simultaneously, like not just one single game—you learn to play Pong, uh you should be able to play Breakout much faster than learning to play Breakout from scratch—transfer learning." So that was the first project I actually worked on. I loved the idea of um um all the papers are DeepMind, and I I would just like literally be in the lab all the time and you know, keep reading their papers, trying to like implement them, um and like borrowing gaming GPUs from other people in the lab uh and and using it to train neuron nets on it. So that was the start, and like you know, I I I really enjoyed those days.
I was thinking before this like, what are the single most pressing questions right now, and what do I most want to ask? And I think the first one that came to mind for me—and when I bluntly had so many people message me when I put out about our show—the first one was one of diminishing returns. And it's when we look at model performance. I think we've always had this kind of belief that you throw more compute and you get much better model performance. Do you think we've gotten to a stage now where we're starting to see diminishing returns?
I think it's a nuanced answer. I can just say no, and you would be like, okay, if I Brute Force still works, but that's not the reality either. Like it's not like if if I suddenly came and say, "Hey, Harry, take my $500 million uh and and and go build a big cluster uh take like uh you know like trillion tokens and and get a model better than OpenAI," it's not going to happen like that. There is still some Alpha left in making these models bigger and training them on more tokens, but you would only get the bang for the buck if you put a lot of effort into like curating the data; otherwise it's just not worth it. Like I know so many research labs—I can obviously mention who they are, but who train really big models on a lot of data and ended up with nothing. It's a lot about what data you train on, how you mix uh English and like other languages and code and like math uh and like all the Chain of Thought reasoning, and then how does it play out in the scaling law like in terms of Chinchilla optimality, and like we later discovered even Chinchilla was not optimal; it was just a guideline. And and then how do the mixture of expert models like like be more computation efficient? All these things matter, and like um that's where I think like you know, those who do it right, those who get these 128 details right are the ones who uh end up benefiting more from more scale, and that happens to be like three or four labs at this point, and you know like I think I'll just give you an example—don't uh uh judge me here—judge Arthur of Mr. uh like like when xAI released the model uh the open source first Grok, and Arthur tweeted saying like uh that's a lot of superfluous parameters because the model was 300B or something—it was not even as good as the MS uh like like like 7 * 8, 56B. So you could you could train a model that's like 6X larger and end up still with the worst model. You could have spent a lot more like money and ended up with the worst model.
If you talk about the kind of curation of data there as kind of the central factor in terms of determining quality of performance, I had Hoffman on the show actually earlier this week, and he said actually that we will see the kind of verticalization of models, that you will use different models for different things. Is that where it leads to then? Is that what you're pointing towards?
No, um and I I actually think that viewpoint is flawed. Uh I I used to think that'll happen too, but I can give you another example that kind of defeats that purpose. Um Bloomberg spent a lot of money training Bloomberg GPT. If you remember, uh they they went, wrote a paper on it, saying they trained their own foundation model, and that model uh is is beaten convincingly by like a GPT-4 on all the finance benchmarks.
How do we know that's not case specific? It could be they just blly approached in the wrong way; they didn't have a good enough team; whatever that is. It doesn't necessarily disprove verticalization of models, does it? You know, could have had a better team; they could have ended up with a better model. No questions about it. The the the the question I'm trying to pose here is that what is the magic in these models? Where is it coming from? These models are magical. Like you're you're not training them for what you're using them at test time. Like the way you prompt and use these models as if they were human in the chat window is not what they were trained to do. They were just trained to predict the next token on the internet. Sure, they were fine-tuned a little bit to be good at chat, to be good at instruction following; all those things definitely, but that is just a very small amount of compute that was applied to these models. So what makes these models magical is the general-purpose emerging capabilities, the fact that they can do things without being taught how to do it, or they can catch things on the fly with some little bit of prompt instructions. Now that doesn't come from any domain specificity; it comes from from the emergence of having training on so much. These neural nets are amazing that if you just throw very diverse sort of data at them, uh they they pattern match on the abstract skill required to be good at all of them at once, and that abstract skill, that that abstract IQ is what is making these models amazing for you on practical production use cases. So when you are saying, "Oh, I'm just going to go and make a domain specific"—how many tokens do you even have in the domain? Like think about it. Code is probably the only domain that actually has a lot of tokens. Um you can throw like a lot of Enterprise data at a model and say, "I have a lot of internal data that nobody else has," but that doesn't mean that these models will absorb a new kind of reasoning that they couldn't get from the internet. It's very it's like one of those things that very few people understand. Why are these models even good at reasoning? It's not well understood. Is it because they're training on math? Is it because they're training on code? And even that is not well understood today. Like had you trained a model on just um textbooks, would you have not gotten reasoning? Like these are questions that we don't yet have good answers to.
Do you think models are good at reasoning? One, and then I think like a breakthrough in reasoning will be one of the biggest kind of um breakthrough moments in the next wave. How do you feel about where we are today in terms of quality of reasoning and what is required to break through in the next wave of reasoning quality?
It really depends on what you call as being good at reasoning. Are they better than an eighth grader? I I think so. Uh are they better than like a 12th grader? Are are they better than like 75% of the 12th graders? Most most likely. Uh are they like going to win the IMO or IOI? No, definitely not. So there's like a spectrum right of people good at reasoning, even among humans, and I'm sure like AI is like somewhere like in the median right now uh of like high schoolers. Um can it get to like a median college undergrad? Definitely. It seems like we're on the pathway to getting there. Would it be like talking to um Faraday or Einstein? Not not anytime soon. Uh I think that's what people are—some people call it as artificial super intelligence, like like uh really the mu plus 7 Sigma sort of people on the planet—and like I think when we achieve that, it'll break all this $20 a month business models. Like would you—I know like you know businessmen in the past—like have you watched this movie uh Prestige?
No, I haven't.
Like the movie like like, uh, where the there's like magicians you know, competing with each other.
Okay.
In that like there's Edison and and this magician wants to steal a trick from Edison on how to make things disappear—sorry, not Edison, Tesla—and and uh he's willing to pay like a lot of money just for the one trick. And I think that's the sort of thing you would get to—model got really good at reasoning where uh just for for the output alone is you're not even paying for a monthly subscription; you're paying for one single session, one single chat, one single output; you would pay a lot of money. You're an investor, right? If I literally told you which company—"Hey, Harry, listen, I got all the inside information; I know all the revenues blah blah blah"—if I came and told you uh you know, or or let's say even if I didn't have any insider information, if I was such a good reasoner and I gave you this—"Harry, this is going to be the set of companies that actually matter two years from now," and I gave you such amazing reasoning that you probably would have had to spend like two months talking to 100 people, then would you have paid like you know, 10K for just that answer?
You probably pay 10 million.
Exactly. So even if you pay 1% of the ROI, it'll be worth it. Or if this if if say uh people at the level of say Demis Hassabis, they want to look—they they're like incredibly smart—who's going to advise Demis? You can count the number of people like in your hand, right? Um and if Demis feels like there's an AI that can advise him, what's the value of that AI? It breaks all your mental models of like $20 a month. I think that's what is lacking. If you say, "Do we have true reasoning?" And if if if the benchmark for true reasoning is an AI that can advise Demis Hassabis, we don't have that today, but there are AIs that can advise uh a person maybe making 120K a year in UK. Uh I think we can get there, but like this is where like you you got to like clearly be precise on what good reasoning is.
I understand that in terms of the precision around good reasoning. When you think about the trajectory of reasoning quality—I know it's a question, forgive me for it—how do you think about the timeline there? Do you think it goes up, flat, up? Is it a continuous gradual increase? How do you think about the trajectory and slope of reasoning improvement?
I don't think we know the secret sauce yet. At least according to writers, the M—the the the news media writers—they claim like OpenAI has some new thing called Q*, you know, it came out during the whole Bard saga, um and like that's the sort of thing that they're working on to like make these models like use their own data to bootstrap and make themselves more intelligent. And then um xAI recently hired this guy um Eric Zelikman from from Stanford who's written these papers on something called the STAR—Self-Thought Automated Reasoner—um or like self-thought reasoner, where like it's basically taking the model itself, make the model explain its own outputs, and then um whatever is the right output, you train on that; whatever is the wrong output, you take the right output and then you ask the model to explain why that was right and train on that. So you basically training on not just the output but also the explanation that was used used to achieve the output. And if you can do that, you're basically training a model that can think and reason and get to an output, see if it's correct, go back, reason again, and iterate. That is what is lacking in today's models. Today's models are just giving you the output. Tomorrow's models will start with an output, reason, elicit feedback from the world, go back, improve the reasoning, and until they converge they'll keep on trying to improve the output. And I think when that is achieved—I don't know when that's going to be achieved; maybe it'll be achieved in a year or two; maybe it'll take three or four years—but I think when that is achieved, that is the beginning of, I would say, the real reasoning era, where we'll figure out how to make these things more efficient. We'll be allotting a lot—the only only problem here I see is this is a game that won't be played by academics like before, because just to do the inference compute, to do all these reasoning—like getting an output, going back and reasoning, uh building a rationale, then going back and getting another output—just to even do this process takes you a lot of inference compute; you have to pay money for this; and so even a single experiment costs you a lot of money until you arrive at the truth of the algorithm, and then that algorithm to run it is going to cost you a lot of money to get all the data, synthetic data to train on. So I I feel like this is where uh companies with a lot of capital are going to be way more more advantaged to pursuing this research. And so if at all it happens that there are only like four or five contenders to do this, and uh whoever ends up with the algorithm the first has a massive advantage, because it seems too good to be true sort of thing where once you crack it, you can just keep throwing more compute at it and like get a big lead over the other models.
So we are absolutely going to talk about kind of the funding required to to finance these models. I do just want to stay on performance and capabilities. Why is it so difficult to have models with memory? Everyone says, "Ah, memory is the challenge." I don't understand why. Can you help me there?
There are two things here to consider. What what does memory mean? Is it like a sufficiently long context that's practical for most use cases, or is it infinite long context? Like basically there's an AI for Harry that uh remembers all your life, every single aspect of it, every single detail—that is like infinite memory—and I think like we don't even have the algorithms for it yet today. And then there's another AI that's sort of like—it's like Gmail sort of a thing where you know, it starts off with like a sufficiently large storage that like like is practical enough, and it keeps—it keeps expanding over time, and then now it's like throttled beyond which you have to pay $10 a month or something, right? That seems more like where we are headed right now, like like people are expanding the token window from 128K—like started with 32K, then it goes to like a million, then DeepMind announced 2 million. So so I feel like that is already good enough where at least we can prioritize and throw out like what's not relevant and like keep keep keep using memory. And as you said, that's not very hard to do. There's one small challenge there though—it'll be figured out—but today's case is that we have achieved long context before achieving good instruction following. So you can dump a lot into your prompt; you you have the memory, but models can hallucinate or get confused because of so much information to focus on. You need to ensure that the instruction following capability has no degradation despite adding all this long context capability. I think that's not the case today, which is why like these models are not so good that like you know, they can just write an entire code base yet, but all that will happen. I think I think I think it's it's just a matter of time before you know, they run another training run and like figure out all these bugs. But the second thing, I'm not sure how to do like infinite context. I'm not sure.
When we look at the different foundation model providers, I do just want to kind of move to the ecosystem itself, and before we touch on the funding itself, I just look at—and everyone says that you know we're seeing the commoditization of foundation models—as you know, I'm interested to hear your thoughts. How do you see the end state for the foundation model there? Like are they getting commoditized as people say?
I think today the word commoditization is sort of true. Uh I mean like it's sort of true in a sense. A model that's like 75%, like GPT-3.75 level model uh is commoditized; there are like too many models like that today in the market, some open source and some closed source. I think GPT-4 quality models are not yet commoditized; there's only probably one or two alternatives for the people today, like Claude, Opus, or some Gemini. Let's say if it's just like two or three alternatives, it's not—I wouldn't call it a commodity yet—but will it be commoditized? I think so. But by the time it gets commoditized, would there be a 4.5 or 5 that's way better? TBD. The training run is happening. My my prediction would be there would be another great model after 4 that's like very good. Like I wouldn't say GPT-4.5 is like a lot smarter than GPT-4 Turbo; it's it's more reliable; it's it's it's better; it's faster, cheaper, but it's not like how 4 blew 3.5 out of water, that sort of thing. If whether 5 can do that to 4 would answer your question of whether these models are getting commoditized. This is not like a bad business unlike any other before whereby you every six months have your core product basically made redundant.
Is that true though? Like I saw your interview with Altman and like like like the Brad Lightcap, but um is it actually true that like a product gets redundant because a model gets an upgrade?
I think so.
Is it not like—which GPT-3 is GPT-3 not like redundant now that you got GPT-4?
Yeah, but your product is never the model. Um there's like—Okay, so let's maybe decouple this. If there are companies that are working on foundation model competitor OpenAI, it is definitely like one of the worst arenas to be part of—almost like I think there are five men standing today sort of a thing—um Google and Anthropic, Meta, uh Mistral, and and you can say uh uh to like after X new funding round, maybe you can include them too, but that that's a game that like is so hard to play, and I'm very impressed that Mistral is even in the arena with like 10X lower capital than the rest.
Are you not in that same arena?
We post-train models; we post-train them; we're not foundation model trainers. For example, we can take any model that's there on the market today and shape them to be really good at what our product does, including like open source models and like making them really good. Have we trained a base model? When you say that's like a Llama 3, 70B, there's a base model that just trained on predicting the next token, and then there's the supervised fine-tuning and RLHF steps that train them to be very good at chat and like instruction following, summarization, and like translation—all these skills. Now that the second part is what adds magic to the product; without that you're not going to have these good chatbots, but the first part has the base IQ that builds the base IQ for these models. Are we not doing the first part? It's very very like—it's a losing game almost to play the first part because every time you end up finishing a large training run, you you burned a lot of money; you have a great model, and then you watch it be like destroyed in the leaderboard by the next update, and then you have to again catch up; you go spend more money. So where how are you recovering all that money back? You may recover that through the APIs, right? But if nobody wants to use your API, if somebody else is just offering a better model at a cheaper price and faster, that's why I think it's a hard game. Now it's not a hard game because it's hard to train these models—sure, the science behind it and the people difficult to assemble—but ROI-wise, like business-wise, it's very difficult to compete here.
Is that not what we're saying though about the commoditization of models being when you get to a stage—oh, everyone's at that stage; we have to do the same again; we have to do the same again; same again—your your model becomes redundant.
Exactly. So that's why I said I think the second-tier models, that the models that are not the most cutting edge but cheap enough to like operate a business on top of it will get commoditized, but there will be some frontier models that are like so smart or like so much better than the second-tier models, and I think those are still—that's still a game being played by like three or four people today.
Okay, it's a game being played by three or four people. Does that end as three or four people, or does it end as one person?
I think the answer to that really lies on uh who cracks bootstrap reasoning, you know, the thing we talked about a little bit earlier about models using their own outputs to reason and improve. Whoever cracks that first, if they allocate all their capital on just scaling that up, I think it'll end up as one person. But if they're hedging, hedging, hedging, it won't end up as one person.
Who do you think that person is most likely to be?
It's likely to be uh OpenAI, Anthropic, and I I can make a good case for both of them. Um OpenAI because they're far ahead in terms of lead; they had in doing these things first. Anthropic because they're algorithmically a superior company; like they got whatever OpenAI got to with lower capital; they have better post-training and like like things like that. So OpenAI is on the other hand like extremely like like advantaged on capital and uh speed. So it really is like a question of who you know which matters more: is it is it clever brains and like some amount of capital, or is it good brains, a lot of aggression, and a lot of capital? If it's a second, it's OpenAI; it's the first, it's Anthropic. If OpenAI didn't have the Microsoft partnership, would you choose Anthropic?
Anthropic also has partnerships—Amazon and Google—MH. So that's why I'm saying they're both reasonably like at an approximate level, equally well-funded organizations. And now xAI also has like a pretty good amount of money and good talent, but they're playing very far from behind in terms of timelines.
I think that you're going to see the kind of large cloud providers realize that they need to acquire these models in different forms, and they will continue their core cash cow businesses as cloud providers, but they will acquire these models and add them in as complimentary features that they already provide, uh and you'll see your Anthropic, you'll see your Coheres, you'll see your Adapts acquired or acquired by these large cloud providers. Do you agree with me in that prediction of the next three to five years in terms of how it shakes out with those acquisitions?
I don't think so. I don't have a prediction on Cohere, but um I think with OpenAI and Anthropic, the value of uh the value of those companies is not in the models they have; that is a very uh first-starter approximation. I think the second-order approximation is it's in the machine that's building the machine—that specific group of people with all the tacit knowledge required to train these frontier models and innovate algorithmically—and what is likely to be the real reasoning breakthrough, and the accumulation of compute they have is the reason why they're valued at this price where the revenue and the valuation make no sense, but they are because it's—I always—I think about valuation is like how easy or difficult it is to reassemble this whole thing, and and and the thing is not just the output; the thing is also the machine that gave you that output. When you say models are getting commoditized, so OpenAI, Anthropic are not that valuable, I disagree because these are the same guys who would produce the next model. Are those guys getting commoditized? Like the the talent?
No. In fact, it's getting the opposite of commodity; like they're all being paid a lot of money to stay in these companies, and so the knowledge only stays with them because people don't publish anymore. There was even a joke—I I recently got to hang out with one of very great researchers—I even made a joke that the best research is the one that's not being published today, and like so there's nothing to read on arXiv anymore. So even the guys at Stanford who wrote all these reasoning papers, they now—like Musk paid them a lot of money to work for for him, so he's not going to publish anymore. So you—it does no—that's what's happening. The commodity is not in the model; the commodity is in the people who produce the models, and that's not a commodity yet. So that's why I feel like these companies are valued a lot, and and they have so much leverage that they won't get acquired because uh like like you're not like like if these people don't want to go work at a big company, and the big company needs the output to keep doing their business—like Microsoft needs GPTs to sell and make sure the number one cloud—AWS needs cloud to to sell to make continue to retain the lead in the cloud market—so uh they have no need or like desperation to get acquired, uh at least the first two—OpenAI, Anthropic—I don't think are going to get acquired. The flip side is that um models—they don't they don't produce any breakthrough like scientifically; it's not possible to keep cramming more and more tokens at this and keep seeing the Jews. That's when what you said is likely to happen. If like say after even one year OpenAI doesn't have a better model, yeah, then the leverage goes away because it's over. This—you've got to actually produce a new thing, and the people are unable to produce it, so their value goes down, and we have to play it out. I I think we both could like both of these things could be true. I think these guys will still produce breakthroughs, so I I that's why I have a different prediction, but time will tell us honestly who's right.
Can I see—we mentioned kind of access to capital, that obviously OpenAI slightly more than Anthropic. You know, I think the thing that struck me was when I heard that you know Mistral's new funding round in terms of size was about 30 hours of Microsoft's free cash flow. You—Microsoft do $330 million of free cash flow per day. In a world where that is the case—I not cynically but just genuinely—how does anyone compete? Like you know, being blunt, we can take this out. You you're rumored to be raising, and it's rumored—you don't need to comment at all—like the amount that you raise relatively is just insignificant compared to Microsoft's free cash flow. How does one compete in that world?
That's why you got to build a business, right? So first of all, let's let's separate the two things. If Microsoft is generating that much cash flow, uh why are they not able to poach uh all the OpenAI scientists or Mistral scientists to come work for Microsoft? Like they could take that money and and and ask one of those people to like ask like 10 of those people to you know, come work here, and I'll pay you a lot of money; you know, you don't no longer need to work at OpenAI; just directly build the AI here; whatever GPUs I'm giving for OpenAI, I'll give it to you directly. It's not happening right for a reason—that it's like people want to work with other best people. So it's it's not enough.
To get one person you want, you have to get the whole thing. Uh, that's why there were all jokes, you know, when the whole board drama was happening, that Saia acquired OpenAI at a small price because he got the whole team. I, I think that's the uh difficulty here. It's it's cash flow; it doesn't change the dependence uh issue.
If they can get these models from people other than these two companies, yes, that changes the equation a lot. Like they can just uh um, you know, like get it from open source and sell the same models and like make the same amount of money with less spend, then that is bad news for the foundation models.
Um, as for like what is the way out of here, I think like you got to build a business yourself. Like it's fundamentally every company that raises capital has to eventually build a business or uh hope that like their algorithmic prowess keeps staying forever. I would bet on like those who are serious about building a business. Like OpenAI is building a business, for what it's worth. I think they they have like what 2 billion in revenue annually, which is like higher than Snowflake or at least like as good as Snowflake, you know. So they're not as capital efficient as Snowflake, but they are, you know, in the same league in terms of recurring revenue and growing faster. So that shows you that like, you know, if you are serious about not just trading these models but also like getting it to the market through products and making revenue out of it, there is a potential for you to like be independent and self-sustaining.
Are you focused on building a business today? Yeah, you said we're going to move away from the 20 pounds per user. It's exactly what you are—20 pounds per month. Um, I know I'm a customer, and I don't think that business is actually that good. Why? It's not high margins enough. If you can get to like a YouTube level thing, sure, you know. I think um, Netflix, YouTube, sort of like user base, you know, 50, 100 million people paying for you—yeah, definitely that's a great business. But I I don't think um we are at a point where these AIs are so fundamental to people's lives that like 100 million people are subscribing to it.
If they can get there, if you can build a product that's not just AI but has a lot more things to it, and people pay a lot of like the monthly fee for it, and the retention is like close to 100%, yes, that's a phenomenal business. And I think we we will try to do that too. But uh, all these great subscription businesses are also doing ads for a reason—margins, right? You want to you want to like look—whatever we criticize Google for, the greatest business model like in the last 50 years is that click-base advertising. It is this insanely good business model—uh, 80% margins.
What was the internal discussion with you and the team when you were talking about adding advertising as a monetization engine? Just take me inside that conversation. How did it go, and how did it net out? You know, there's this whole Larry and Sergey PageRank paper that said like advertising is fundamentally uh incompatible with like serving good results to the user in a search engine. I truly believe that, and like I've read books that said like they pushed back on introducing ads as much as possible until they gave up to investor pressure. Um, we were like, "Look, let's be practical. This is the most highest-margin business model ever invented, but let's do it in a way where we don't have to be as high margins as Google. Like you don't have to aim for that 80% margins. Like as long as you can get a good, reasonably good high-margin business without uh failing on your duties to the user, be happy. Like don't be greedy." So that's our that was our thinking, and we didn't actually have much debate on this. It was like, "What is the way to do ads without corrupting the user answer? Uh, without without making as in you make sure that uh the answer is not like influenced by by uh the ads, or the links that you cite are not influenced by the ads." And if you can ensure that, I think it's a great I think it's a great uh idea to explore. That's why we have other other surface areas for ads too, like even the Discover feature and Perplexity, which has like, you know, a bunch of threads, interesting uh threads every single day to like read. Uh, that's just going to be like an endless scroll at some point, and and and Instagram does ads in that format. They do ads in their format. So ads is a great business model, and when it's relevant, it's amazing. Like I've literally not met one single person who came and told me Instagram ads suck. It's actually pretty good. It's all about cracking the relevance code. Like if you crack the personalization and relevance code, AI is like pretty amazing.
Do you think you've cracked the relevance code? We—I'm saying we want to try. I'm not I'm not saying we have cracked it. And if you cracked it, I think we should be worth way more. But but but like we—mean, first of all, it it can only it's like a chicken and egg problem. It can only be cracked when you have a lot of users. So advertising is one of those funny things where there's no way it can work well when you don't have a lot of users, and then when you have a lot of users, it can work really well if you get all the details right. I was talking to Mark and recently once, and he told me how like in advertising it's like three tiers, where like the top tier is like Google, and then like one and a half like like one is Google, one and a half is Meta, because even between Google and Meta, Google benefits from every other advertising other people do, because at the end, once you discover the brand, you go to Google and click on the link they have. It's amazing like how they benefit from everyone else's hard work all the time. And then there's like companies like Twitter and like Reddit and Snap, which is like 30 or it's like and and he said the gap between these two so high, this is like almost climbing the peak of the mountain, and this is just like somewhere in the bottom. So that that that is the extent to which ads have been dominated by like Google and Meta at this point today. My point is that if we can get the fundamental mistake that Google made right in our journey very early on, where we're not overly greedy on one source of revenue and are diversified enough through subscriptions, advertisements, APIs, Enterprise, I think we have a chance to build something that uh achieves the alignment between shareholders and users a lot more. Like Jeff Bezos has code, right? That basically the shareholder and and and and the user should be aligned. If not, then you don't have a customer-focused business. This is where Google got it wrong because asymptotically they couldn't achieve that alignment between the user—that is you using Google—and the shareholder. Wall Street allows it when Google puts more ads; you hate it.
You mentioned OpenAI is 2 billion in revenue. Um, a lot of that is Enterprise, and they've built out Enterprise actually incredibly well. You kindly mentioned my show with Brad where we actually kind of touched on it. How do you think about when's the right time to build out Perplexity's Enterprise division? The number one insight that motivated us to build this was, "What is the most used Enterprise tool today?" Google. I mean, email also is G Suite and as part of the Enterprise offering. Okay, Google, that's right. Like you search every single day at work; all the data is something internal to your company, like as in the specific queries, but nobody cares because you need it. You cannot live without it, and and uh you pay for it through your time, and you pay for it through your data. Um, now this this thing changes in the AI-native search world where people always are they're always worried about data leaking to a—they don't care if a leak to a traditional search engine, but if the search engine now has a lot of AI in it, they they're worried. So we said, "Okay, if you want to use Perplexity at work and your employer doesn't let you use it, we'll solve that problem for you. We'll offer an Enterprise Pro with compliance and security and data governance and literally offer you the same product with all these uh security features." And that became our Enterprise Pro. Now that's a start. You need features too that are more catered to the Enterprise than just the consumer, and that's what we will build, and we want to build it in a pretty differentiated way—rethink what even internal search means. Like not just can build pipes to every single like Enterprise tool like Slack or Notion, but really think about like like what what is like the ranking problem. Why is it hard for the Enterprise compared to Consumer? And like if we can build like one UI where all the proprietary data, external data, internal data, all the different models, open source, closed source, live in like one single platform, and and and you know you can take your output, convert it into good readable pages, organize it by book like as a knowledge base, index it yourself, like that can be a good Enterprise offering. And I think we'll work on that. I'm not saying we'll succeed at it, but we we we'll try to do something.
With total respect, my friend, are you nervous about building out an Enterprise product? When you look at the GTM, it is a very different motion. Enterprise is a is a big beast to get your head around, um, and it's a challenge. You you said there about the scale of OpenAI's sales team. I know I've got many friends in it; it's a big thing. How do you think about getting your head around the GTM building exercise of an Enterprise division? Do people buy Perplexity Enterprise and OpenAI Enterprise, or either? All my sense is that like AI is still so early today that nobody's locked in and loyal to any any particular uh Enterprise tool in AI, and none of them even have a lock-in effect to like make your data live on like one single uh to like—it's not I'm not even talking about things like why is it hard to migrate from Snowflake to Databricks because the SQL format itself is so different, and once you wrote all the SQL queries in one format, it's so hard to change. Like like it's not even things like that in AI. Like your custom prompts that you wrote for ChatGPT can be taken over easily to Perplexity. It's very easy. So um, I I think Enterprises are still willing to tinker and experiment and try different tools. And that said, if there is no differentiation, they will win in the beginning—the one with a bigger brand and bigger team has an advantage. But is it game over? No. Is this game begins today? I think like this is exactly the whole wrapper thing, and I um if the value you add is like uh very little on top of the model, or the model is the one that's adding most of the value, and all the stuff you built around it don't matter, yes. But if you build enough value around the model that is very difficult to do without coordinating a bunch of other hard-to-achieve engineering feats that are not just LLM-based or like have a lot of human element involved in it, it is difficult to see a world where like like that is not valuable and people don't want that. You know, the specific search thing—why is it that like um Google AI overviews was bad? They have the world's greatest index; they have the world's best models too, but it wasn't good enough. Or why is it that uh people still think, at least a good chunk of people—I'm not saying everyone—think uh ChatGPT browsing is not as good as Perplexity, despite them making so many updates over the last one year? Why is Perplexity browsing better than ChatGPT? I think it's just a lot of small details. I'm a big believer in those who can orchestrate models and data sources and build great UX and and keep innovating here all the time will survive this whole wrapper argument. I think it's just like uh going to be difficult until you build a business. Everyone's always afraid you're going to die, but as you are accumulating the users and as you are figuring out the business, it feels to me more like like the biggest beneficiaries of the commoditization of foundation models are the application layer companies.
Why is that? If models get commoditized, then the price of the models goes down, and then those who directly reach the user using those models, harnessing the power of those models but packaging it into like great product experience and utility value and directly own the relationship with the customers, the users have a lot more advantage because they are able to like take something that's a commodity and sell it at a premium, which is a great business. If models get commoditized, I'm happy. If models don't get commoditized, I still want to figure out a way to benefit from that, and that's why this is a great difficult company to build. It's not something where we just hire an SVP of product from Twitter or Meta and ask them to figure out product for you. It's not easy; they don't have the mental models of like what happens when the next AI model is so much better, how to rethink the whole product strategy. Similarly, it's not something where uh you hire a great AI person and ask them to like build product because they're always going to think the model is the most important thing and keep trying to do everything through the model. You need the right sweet spot of design and product and AI and search all together, and that assembly is not easy. Like it's pretty hard, and that's why we are able to do things as a wrapper that other people are not able to do.
Have you been surprised by the fundraising process? Fundraising processes are brutal. I think most people think like you just go to like—there's always these memes about about like, oh, if it's an AI, people are just like willing to write you the term sheet without even doing any diligence. Well, like, welcome—why don't you try to raise? It's pretty difficult actually. Um, everyone's asking all the questions that people on Twitter roast uh wrappers: What happens if uh OpenAI does this? What happens if Google does this? What, you know, why would they not stop giving you models? Like, um, how will you build your own models? Like, how are you going to ever build a search index that's like really good? Or um, you know, how do you compete on the Enterprise sales? Like, like all these are questions everybody asks, and like and when you don't even know a when you don't have a good model of the future yet, you have to give them good arguments. But at the end of the day, it's all like arguments; nothing is there. And one thing that we do have in our favor is like a good track record of execution. We've been around for like less than two years, and the amount of things we've shipped is quite a lot compared to the team size and funding we have.
Of the cash raised, how much goes to compute? Like like 50%? Like 75%? Just uh—I don't have the exact number, but like, first of all, let me let me give you like two things: We have not spent a lot of money uh—what I'm I'm—it's not like most of the cash we raised has already gone away to compute now, but what I'm saying is whatever money we spent, the majority of it has gone to compute. I don't have the exact percentage, but the majority is compute, and the computer is either as buying GPUs and serving models or post-training models or uh money we pay for uh APIs like Anthropic or OpenAI. That's fine as long—so that's why it's very advantageous to us to not train our own foundation models because if we were doing that too, most of the funding would have run out because the way it works is you have to pay three years in advance to get a big cluster. Like you have to commit to that. It's not like all the money goes away immediately, but you have to commit to three years to get like, you know, thousands of GPUs at once if you want to compete in that game. On the other hand, because we we're not doing that, and we benefit from any commoditization in the models, and when it kind of happens, we have all the money to go get users like and and getting users not not simply through like marketing but actually more in the Amazon Prime sort of way—giving a lot of great features at like amazing prices, getting to retain you through superior product execution and doing it intensely over like a sustained period of time, and then like, you know, building like sufficiently large user base and brand loyalty that that that is the model that we we are going for. And in in such a world, like advertising can be pretty powerful at that scale. Every business has a core monetization engine; they have ancillaries, but there tends to be one which is dominant. When you look at, you know, uh Perplexity in five years' time, what is your dominant engine? Is it consumer subscription? Is it advertising? Is it Enterprise? I would predict it'll be advertising if we crack it. Yes, it'll be advertising. If we don't crack it, if we are not if we if we don't if we haven't grown to that level in user base and or if it grew and didn't figure out how to advertise really well, I think it'll be the other two. Either way, we can be profitable. I think with advertising we can be really, really profitable. And then you can ask him, "Hey, Arin, why do you care about profits?" Like Sam Altman doesn't care, but he doesn't care because he's not interested in actually just focusing on product as a business. Like he's trying to build AI, and like he already told publicly in an interview that, you know, even if we spend like, you know, $50 billion on AGI, it doesn't matter. So that's a different company. We shouldn't be seen as an OpenAI competitor at all. We're not an AGI lab. We we—you can say Perplexity and ChatGPT are products in a similar space, and and there's like some competition for mindshare and users, but even that will like be pretty clear like two years from now. You're not going to keep asking how is Perplexity different from ChatGPT today; you are, but two years from now, I don't think so. If that's still the case, one of us is just copying the other.
What do you think is the best question you are never asked? You've done interviews before. I think someone asked me um like why are you doing this sort of thing, and this is sort of a question where you don't actually know yourself. I think a lot of people give uh these um made-up answers like, "Oh, I had an existential crisis. I needed to save humanity from extinction. Uh, like, you know, I need to preserve the light of consciousness, and so I thought about what are the most important—" Like these are sort of things that I've seen entrepreneurs say, but uh reality is like like you just sort of look up to some people; you want to be like them, and you try to carve your career path according to what they have done, but then you end up like figuring out there are things that you really like, and you shape it to the style you want. And at least that's how it's been for me. I have been a big fan of Larry Page, and I always wanted to do some things that of that of that scale of ambition, but that was not the reason we did a search engine, though. Like we we started with something else completely. So that's a question that I actually don't have a clear answer to, but I really like the question because it's a question worth asking yourself constantly: Like why are you even working on this? Like uh Steve Jobs has this thing, right? Like if you uh felt like if you if you internalize death, if you normalize death, and and and every day morning you stood in front of the mirror and asked if today was my last day, would I still be doing this? And if the answer to that question is yes, go and give your best a day. If the answer to that question is consistently no on a regular basis, you really have to rethink your life priorities. And for me, like Perplexity is yes, like hell yeah. Like every day, even though it's painful, even though it's um, you know, stressful, takes a toll on mind and body, I think it's worth it. You still looking credit young, so don't worry; it hasn't aged you, Arin, so all good there. I'm hiding I'm hiding my gray hair very cleverly.
Listen, I do want to do a quick fire round, so I say a short statement, you give me your immediate thoughts, and I'd love to start on what you changed your mind on most in the last 12 months. Long-term view on people—seen some people like not immediately hit the ground running but give them sufficient time, they are able to like truly transform themselves. It's something that I didn't have the right attitude towards in the beginning, where I always thought like those hit the ground running immediately are the best, but you know, different people have different styles of showing their true Downs.
What's the biggest misconception in AI today, do you think? Short-term thinking. Like anytime somebody comes up with an update, everyone's like the other company is done; like this is over. Um, but that's like, you know, I would say say the the usual Twitter mob, but I would say the biggest misconception among even the more uh well-informed people because the majority of the people in the world are not using chatbots; they just think this is a bubble. They're going to get really surprised that it's not a bubble; it's not overhyped; it's actually underhyped. These things, when taken in the right workflows and UI factors that you're already familiar with, will have a lot of impact. Like chat UI is a new UI; we're not used to using it. We're all used to using WhatsApp and Signal and all that, but that's different; it's not exactly a chat; it's more like a texting service. Then on the other hand, word docs, Gmail, Google Search—like I'm not even talking about the specific products but more like the UI factors of usage, the UIs you're very familiar with—and when AI presented to you in that sort of a format where it feels so obvious and natural as a workflow, it'll have a tremendous amount of impact, and it's not really happened yet.
Have you seen WhatsApp's integration? I have. It's not the right way to do it. Why? I'm not going to WhatsApp to search for anything; I'm going to WhatsApp to text people or reply to, but my WhatsApp most of the times is just having like 20, 30 notifications, and by the time I'm done with them, I just want to get away from the app. I'm not going there; I'm not clicking on on the pressing on the WhatsApp icon to like search for something. Same thing on Instagram; I'm just going there for pretty pictures; I'm not going there for searching about like who's who's won the NBA. It it it's just the user intent behind opening the app matters a lot. This is the same reason why they failed multiple times at doing uh stories and Reels. Story started off as a way to copy Snapchat as a separate app first; that didn't work. Then they tried so many different variants. What really ended up working is the top bubbles and like you, and that only works because you you're starting with the existing user flow; you're already going there to check out other people. So you you have to really think about like not just like why this feature is added but what is the existing user intent in your app and how can you make sure the new feature you're adding ties into the existing intent. That's very important.
What's your vision for the future of browsers? I think you can reimagine the browser when agents start working. There's a reason why we never did a browser. I don't think the browser is going to be disrupted because you get answers instead of links. People still want to browse and and get to a new website, get to a specific website, enter details, fill up forms—all those kinds of things that's not really getting disrupted with the traditional chat UI is just because you can type in like on Perplexity on the search bar. Let's say that integration is done; I don't think you allow the browser more or something. It's it's going to be more productive, but you need the traditional browser functionality a lot. What will have changed though is you go to a browser; you just say, "Start the podcast, Arin," and it already knows exactly um, you know, Riverside; it has to go like fill up your logins and and get to exactly, and then after that it's just over. That would be amazing. That that would that would change everything. Like or like, "Buy me this thing on Amazon," like like it's it's sort of like um completely. Then you can go a step further and say like, "What is the future of the OS? What's the future of Mac? What's the future of Windows?" And so the browser is just an OS too, right? What do you think is the future of OS then? I mean, something like the Her movie can work—like, you know, not talking about the voice but just the OS itself being an AI, completely AI-native OS. Like it's not organized in a traditional way, and you just talk to it, and it just works for you. I think that that that's amazing uh vision to have, and uh that's the sort of thing that doesn't work today. Like like GPT-4 or cannot cannot do it yet.
What's the hardest element of your role that people don't think about or consider, do you think? I think it's just um dealing with contradictions all the time. Um, I I believe the brain is not very good at dealing with contradictions; it actually tires us out when we can't arrive at a convergence point on something, and a startup CEO is all about contradictions. Should you take a risk or should you like double down on what you have? Should you uh move faster or should you set up the company in a way that it can scale? Is it time to like, you know, try out this feature just because it's it's it's uh not something your competitors would do, or continue doing what you're doing well, but but your competitors are doing the same thing? You have to like constantly deal with these contradictions in so many different dimensions, and that's tiring.
Penultimate one: If we were to write, you know, we write kind of pre-mortems as investors, a reason why a company doesn't work when we write an investment, if you were to write a pre-mortem on Perplexity today, what is the reason why you don't achieve your goals? Access to compute? Google innovating and killing you? What is that reason? Didn't execute well. I mean, there's a saying that startups don't kill themsel—startup like competitors don't kill startups; startups kill themselves. It's not that Google Drive killed Dropbox; people people say that as an example, but um there was like a great Enterprise business to build, and Dropbox didn't move really fast compared to like other companies like Box. So startups do not kill themselves; so compar—do not kill startup—stars kill them—kill themselves. So if there was a pre-mortem to be written about us, it's like CEO not making being decisive, uh execution of the company not being good, uh lack of focus, inefficient use of capital. So largely comes to whatever decisions are made, the correctness of them, the speed of them, and execution of them, and whether we focused or not. If these things are not true on a consistent basis, uh, yeah, I think I think we would die, and that that would be the pre—
Final one for you: It's 2034; where would you most like Perplexity to be then? If we do a show then, where is the business then? I think I would just want it to be the assistant for facts and knowledge you just cannot live without, and and you can ask me, "What, 10 years later, do do people even want facts?" You know, there's this thing where you have to always ask this question like, "What is going to be true even 10 years from now?" And if you work on that, you're working on the right thing. I feel like even even in a world with a lot of AI agency uh and and less of human agency, uh people would still want to know what's true and what's not true, and so we are working on that. So if we are the goto assistant for facts and accurate information and knowledge, I think we'll be fine even 10 years from now.
Arin, listen, I've loved doing this. Thank you so much for putting up with my my straying questions, but you've been a fantastic guest, and I so appreciate the time.
Thank you, Harry; that was great.