Transcription
Nothing seems so fundamentally difficult that it cannot be solved if the smartest people in the world work incredibly hard on it for the next 5 years. Humanity has gone through the agricultural revolution and the industrial revolution. We are now experiencing another one. We may not be able to give it a name. People of the future will. But we are definitely in the process of great change. The number of individual solo entrepreneurs that this technology will make possible dramatically increases what one person can do. For the first time, opportunities on a massive scale are becoming available to everyone. The very possibility for a much larger number of people to become entrepreneurs is colossal. Adam Mamjat, welcome to the podcast. Thank you, thank you for having me. Lately, many have been expressing skepticism about large language models. Amidst general pessimism, people talk about their limitations, why they won't lead us to AGI, and that perhaps what we thought was a couple of years away is now being pushed to 10. Adam, you seem more optimistic. Share your perspective. Honestly, I don't even know what people are talking about. If you look at the situation a year ago, the world was completely different. Judging by the progress we've made in the last year, reasoning models, improvement in code generation, progress in video generation, it feels like things are moving faster than ever. So I really don't understand where this pessimism comes from. I think there's a feeling that we hoped they could replace all tasks or all professions. And now it seems that automation works at an intermediate level, but not end-to-end, and that labor may not be automated on the timelines and in the way we expected. I don't know what specific timelines people previously envisioned, but if you look 5 years ahead, we'll be in a completely different world. It seems to me that today models are not held back by intelligence at all. The main problem is giving the model the right context so it can use its intelligence. There are also things like computer usage, which are not yet fully ready, but I think we will almost certainly solve this within a year or two. And when that happens, we will be able to automate a significant portion of what people are currently doing. I don't know if I would call it AGI, but I think it will satisfy many of the criticisms that are being voiced now. In a year or two, they will simply become irrelevant. And what is your definition, AGI? I don't know. Everyone has their own. I, for example, like this one. If you have a remote worker, a person, and any work that can be done remotely can be done by a system, then that is AGI. It's not necessary for it to be better than any person in the world in every single profession. Some call it ASI, implying that the system must be better than human teams. One can argue about different definitions, but in my opinion, when we surpass the average remote worker in the work they do, we will be in a completely different world. And that's a pretty useful benchmark for such definitions. So, in general, you don't see the same limitations of large language models that others talk about. They still have huge potential for development, and we don't need a fundamentally new architecture or some radical breakthrough. I don't think so. Yes, there are certain things like memory and learning, continuous learning, that are not easy to implement in current architectures, but even they can, figuratively speaking, be simulated. And perhaps we can make it work well enough. So far, we are simply not hitting any hard limits. Progress in reasoning models is incredible. I think progress in pre-training is also happening quite rapidly. Perhaps not as rapidly as many expected, but certainly fast enough that we will see significant shifts in the coming years. Amjat, what is your reaction to all this? Yes, I think I've been quite consistent and perhaps consistently right. I dare say, consistently myself or with what I'm saying now, and with both, and, it seems to me, with how things are developing. I started to doubt more publicly around the time the discussion about AI safety peaked somewhere in '22-'23. And I considered it important to be realistic in assessing progress, because otherwise we will scare politicians, scare everyone. Washington will descend upon Silicon Valley and shut everything down. Therefore, my criticism of ideas like AGI in 2027, that very report that Alexander and someone else seem to have written, as well as understanding the situation and all these hyped studies, reports, is that it's not science, it's just a feeling, a vibe. That's what, in my opinion, won't happen. The entire economy won't be automated. Jobs won't disappear all at once. All of this, in my opinion, is unrealistic. It doesn't correspond to the type of progress we are observing, and it will ultimately lead only to bad policy. My position is this: LLMs are incredible, truly impressive machines, but I don't think they are equivalent to human intelligence. LLMs can still be fooled. Yes, they may have solved the strawberry problem, but they can still be tripped up by simple one-sentence questions like how many letters are in this sentence. I think I wrote about this recently. Three out of four models failed to answer correctly, and even GPT-5 in enhanced reasoning mode had to think for 15 seconds to handle such a question. Therefore, LLMs are, in my opinion, a different type of intelligence compared to humans. They have, they have clear limitations, and we smooth them out by circumventing them in various ways. Whether it's changes in the models themselves, in the training data, in the infrastructure, and in everything else we do to make them work. And this makes me less optimistic in the sense that we have truly cracked intelligence. It seems to me that when we truly crack intelligence, it will feel much more scalable, and the idea of the bitter lesson will become truly true. You'll be able to just add more power, resources, computation, and systems will scale naturally. Now, a huge amount of manual labor is invested in improving these models. In the era of true scaling, pre-training, GPT-3, GPT-3.5, possibly up to GPT-4. It seemed enough to just pour in more data, and the models got better. And now it feels like there's a lot of data labeling work, a lot of contract work. Many artificial RLHF are being created to make LLMs good at programming and turn them into coding agents. They will do that. I think the news from OpenAI about plans to do the same for investment banking speaks to this. I'm trying to introduce a term, I call it functional AI, the idea that you can automate many aspects of many professions simply by entering each area, collecting maximum data, and creating such environments. This will require colossal effort, money, data, and everything else. And yes, I agree with Adam, in the next 3 months, 6 months, progress will be 100%. Claude 4.5 has been a huge leap. It seems to me many underestimate how big that step was compared to version 4. Claude 4.5. There are truly impressive things, so there is progress, and we will continue to see it. But I don't think what we have now is moving towards AGI. My definition of AGI is more like the classic definition from the world of RL. A machine that can enter any environment and learn effectively just like a human. You can put a person at a billiard table, and after a couple of hours, they will already know how to play. Now we have no way for machines to learn such skills on the fly. Everything requires huge amounts of data, computational time, and effort. And more importantly, it requires human expertise. This is the opposite of the bitter lesson. Human expertise does not scale, and today we rely on it entirely. We are in human expertise mode. Yes, I agree that humans are much better than current models and learn new skills in an environment with limited data. But on the other hand, human intelligence is a product of evolution, which used a colossal amount of efficient computation. It's a different type of intelligence. Since models haven't had an equivalent to evolution, they only have pre-training, which is not as efficient. Therefore, more data is required to learn each new skill. But if we talk about practical consequences, about when the world will change, when the labor market will change, when economic growth will accelerate, I think it will depend more on the moment when we can create something comparable to human intelligence, even if it requires much more computation, energy, and training data. We can just invest all of that and still get software that is no worse than the average person in performing typical work. I don't dispute that. Yes, it looks like a brute-force mode now, but maybe that's okay. Yes. Then what is the disagreement? It seems there is agreement here. Where is the divergence then? I don't think we will reach singularity or the next level of human civilization until we understand the true nature of intelligence, until we understand it and create algorithms that are not truly based on brute force. And you think that will take a long time? I'm more of an agnostic here. It just feels like LLMs are in some sense distracting from this, because all the talent goes there, which means fewer people are engaged in fundamental research on intelligence. Yes, but at the same time, a huge part of talent is now going into AI research, where they wouldn't have gone at all before. As a result, we have a whole industry, colossal funding, and computational resources and people. And frankly, it seems to me that there is nothing so fundamentally difficult that the smartest people in the world, if they work on it with all their might for the next 5 years, cannot solve. But fundamental research is still different, right? It's an attempt to get to the basics, to fundamental principles, as opposed to industrial research, where the question is more often: how to make these systems more useful to generate profit? I think these are different things. Thomas Kuhn, a philosopher of science, wrote a lot about how such research programs over time turn into a bubble that sucks in all attention, all ideas. Remember physics and the whole industry around string theory. It attracts everything to itself and becomes a kind of black hole of progress. Yes. Yes. And it seems he had a thought that sometimes you just have to wait for the current generation to leave for a paradigm shift to be possible. He was quite pessimistic about paradigm shifts, but I still think the current paradigm is quite good, and we are still very far from diminishing returns from its further development. I would bet that many innovations can still be made within this paradigm and ultimately lead to the goal. Let's assume we continue on the path of brute force and automate a significant portion of labor. What do you think GDP growth will be at 4-5% per year, or will we reach 10% and above? What will that do to the economy? I think it all depends heavily on what exactly we arrive at and what we will call AGI. Well, let's imagine we have it. Let's say we have LLMs that, at an energy cost of $1 per hour, can perform any human job. Let's just take that as a theoretical point. In such a world, GDP growth will be significantly higher than 4-5%. The question is, will we get there? It may turn out that LLMs capable of doing everything a human does are more expensive than humans themselves. Or they can do about 80% of what a human does, and the remaining 20% they cannot. But I still think that at some point we will reach a point where they can do absolutely everything a human does, and cheaper. I see no reason why this shouldn't happen. It may take 5, 10-15 years, but until then, we will be hitting bottlenecks, things that LLMs still can't do, or, for example, the need to build enough power plants for energy or other supply chain limitations. >> Subscribe to my Telegram channel right now via the link in the description. I have prepared for you the top three materials that, in my opinion, everyone should know. First, a map of a hundred top AI startups – this is the future in one picture. Second, a forecast from an insider at OpenAI who predicted everything that is happening with neural networks even before ChatGPT appeared. And this year, he released a new forecast until 2027. And third, the most powerful is my analysis of an essay by the founder of Anthropic, who is essentially the second person in the world of artificial intelligence. He laid out step-by-step what will happen in the world in the next 5 years, and most importantly, what the universal AI, which everyone fears or awaits, will be like. Go to the link in the description. >> I am concerned about the potentially harmful effect of LLMs on the economy. For example, they effectively automate entry-level positions, but not expert work. Take, for example, QA, quality control, the system becomes very good, but there is still a long tail of rare cases that it cannot handle. As a result, you have a small number of very strong QA specialists who manage hundreds of agents. Productivity increases sharply, but new people are no longer hired because the agents are better than beginners. And this seems like some kind of strange equilibrium. It seems to me that few people think about this. Yes, absolutely. I think this is already happening, for example, with computer science graduates. There are simply fewer jobs than before, and LLMs largely replace what they used to do. I am sure this contributes. As a result, fewer people climb the career ladder that companies used to spend huge amounts of money on, hiring, training, investing in them. And I believe this is a real problem. I think over time it will create an economic incentive to solve it. Perhaps more companies will emerge that focus on training people, or they will be used more actively for teaching these skills. But in any case, this problem already exists. There is another related problem. Since we depend on expert data to train LLMs, and LLMs themselves begin to displace these experts, at some point there will simply be no experts left because they are all losing their jobs and becoming equivalent to LLMs themselves. But if we really depend on data labeling and expert environments, how can they develop further? It seems to me this is a question that economists should seriously consider. After the first step of automation, certain difficulties arise, and how to move to the next stage. Yes, I think a lot will depend on how high-quality environments can be created. In one extreme case, you have something like AlphaGo. A perfect environment where you can quickly go far beyond human level. But for many professions, the amount of data is severely limited, and there is simply nothing to learn from. So it will be interesting to see how easily research teams manage to overcome this bottleneck. If you had to guess what categories of professions will emerge or grow sharply in the future. Some say everyone will become an influencer, some that everyone will be in care or in government service in some bureaucratic role, or perhaps in education. And as more and more tasks are automated, what do you think people will do? Art and poetry. Yes, at some point everything will be automated, and then, I think, people will truly engage in art and poetry. There is, for example, such a fact. The number of people playing chess increased after computers started playing better than humans. So I don't consider it a bad world if people are free and can pursue their hobbies, if there is some way of distributing wealth so that people can afford to live. But this is still a rather distant prospect in the near future, well, in about 10-15 years, it's hard to say exactly, but yes, I would say it's a horizon of at least 10 years. And in the near future, professions that know how to effectively use AI, that are capable of doing their work better with the help of AI, especially solving problems that they couldn't solve alone, will grow rapidly. There will be colossal demand for such specialists. I don't think we will reach a point where we automate absolutely all professions. Certainly not within the current paradigm. I would strongly doubt that it will happen. I'm not sure if it will ever happen, but certainly not with the current approach. And here's why. A large part of work involves serving other people. To understand what other people want, you need to be fundamentally human. You really need to be human to feel that, which means human experience is necessary. So if we are not going to create humans, if it is not truly embodied in human experience, then humans will always be the generators of ideas in the economy. Adam, I want to react to Anj's thought about the human factor, because you created one of the best collective wisdom platforms in the world, and now you've completely gone into AI. To what extent do you think we will rely on people? And to what extent will we trust, for example, in the role of therapists, caregivers, and other similar roles? Collectively, people have a vast amount of knowledge. Even a single expert, who has lived a whole life, made a career, and seen a lot, often knows a lot of things that are not written down anywhere. This can be called tacit knowledge, but even what a person can articulate if asked remains extremely important. I think people still have a significant role in the world to share their knowledge, especially that which was simply not in the training set of LLMs. Will they be able to earn a full-time living from it? I don't know, but if it becomes a bottleneck, then economic pressure will certainly shift there. As for the idea that you need to be human to understand what people want, I don't entirely agree with that. For example, recommendation systems, those that form feeds on Facebook, Instagram, or Quora, are already superhumanly good at predicting what you will be interested in reading. If I gave you the task: "Create a feed for me that I will definitely read." No matter how well you know me, you wouldn't be able to compete with these algorithms. They have data on everything I've ever clicked on, on everything other people have clicked on, and on all the relationships between these datasets. So I don't know exactly how it will all turn out in the end. Yes, it's easier for a human to simulate another human, and this helps test ideas. I am sure that for composers and artists, this is an important part of the process. They create something and immediately perceive it. A chef cooks a dish and tastes it. This is important. But at the same time, they have very little data compared to the volume on which AI can train. So I'm not sure where this will ultimately lead. That's a good argument. Ultimately, recommendation systems do roughly this. They aggregate different tastes, then find your position in a multidimensional space of preferences and serve you the most relevant content. In that sense, yes, that's how it is. But it seems to me this is a narrower story than we sometimes think. That is, for recommendation systems, it's true, but I'm not sure it applies to absolutely everything. Therefore, if you try to give the best prediction of where the world is heading, and this is not necessarily an endorsement or affirmation that everything will be this way, because, it seems to me, the system will be quite unstable, but the book "The Sovereign Individual" remains, in my opinion, one of the most accurate sets of predictions about the future. Although it is not a scientific work, but rather a polemical book. But the idea itself is as follows. In the late eighties, early nineties, two authors from Great Britain, I'm not sure if they were economists or political scientists, tried to predict what would happen when computer technologies reached maturity. They proceeded from the fact that humanity had already gone through the agricultural revolution and the industrial revolution, and is clearly going through another one now. Back then, it was called the information revolution, now the intelligence revolution, whatever you call it. Perhaps we won't be able to give it an exact name at all. People of the future will invent it. But it is obvious that we are going through something fundamental, and they are trying to answer the question: what will happen next? What do they conclude? That in the end, large masses of people will appear who will be either unemployed or economically inactive. At the same time, entrepreneurs and capitalists will be extremely empowered because they will be able to create companies very quickly with the help of AI agents. Thanks to this, their generativity, their ideas, human understanding of what other people want, they will be able to quickly create products and services and organize the economy in a certain way. Politics will also change, because the modern political system is built on the assumption that every person is economically productive, but when mass automation occurs and only a small number of entrepreneurs and very smart generative people remain truly productive, political structures also begin to change. They write that the nation-state in its usual form is gradually losing its significance. Instead, a situation arises in which states begin to compete for people, primarily for wealthy people. And as a sovereign individual, you can, figuratively speaking, negotiate your tax rate with the state that you like more. All of this begins to resemble biology a bit. I don't think it's that far from where things could go. And again, this is not a value judgment or an expression of desire. But, in my opinion, it's worth thinking about. When a person ceases to be the basic unit of economic productivity, everything must change, including culture and politics. Yes, and here arises a question related to this book and our conversation as a whole. At what point do technologies start to reward the defender, not the aggregator? Or vice versa, when do they stimulate decentralization, and when centralization? I remember Peter Thiel joking about 10 years ago that crypto is a libertarian, decentralizing technology, and AI is, conditionally, communist, i.e., centralizing. And it's not obvious to me that this is entirely true. On both sides, AI seems to truly empower individuals, as you said. At the same time, crypto ultimately turned out to be something like fintech, a stable core, you know? And at the same time, it also, in a sense, strengthens the state. We see discussions at the national level about things like the so-called Chinese scenario, which they are going to implement. So yes, the question remains open: which technology ultimately empowers whom more: the periphery or the center? If it empowers the periphery, then the concept of the sovereign individual seems to really start working. Although, perhaps there is some barbell effect here, where, on the one hand, large acting players become even bigger, and on the other hand, there are these very edges. In general, this is all food for thought. I am very inspired by the number of solo entrepreneurs that will emerge from this technology. It seems to me it radically increases what one person is capable of. There were so many ideas that simply never materialized because you had to assemble a team, possibly attract funding, find people with the right skill set. And now, when one person can bring all of this to life, I think we will see a huge number of truly cool things. Yes, I constantly see tweets from people who quit their jobs because they started making a lot of money using other tools like Replit and others. It's really very exciting. It seems to me that for the first time in history, opportunities are becoming massively available to everyone. And for me, this is perhaps the most inspiring thing about this technology, besides everything else we are talking about. Simply the possibility for more people to become entrepreneurs is colossal. This trend will obviously continue to develop. Looking at the next 10-20 years, do you think AI will be more of a supporting or a disruptive technology in the Christian sense? In other words, do you think the main share of value created will go to companies that scaled up even before the advent of OpenAI? Replit then still falls into the second category, and Quora to some extent too. Or do you think that most of the value will be captured by companies that appeared after, say, 2015-2016? There is a related question: what portion of value will go to hyperscalers, and what portion to everyone else? And on this matter, I actually believe that we are currently in a fairly good balance. There is enough competition among hyperscalers so that application-level companies have choices and alternatives, and prices are falling incredibly fast. But at the same time, there is not so much competition that hyperscalers and labs like Anthropic and OpenAI cannot attract money and make long-term investments. So I think the balance is quite good right now. We will see the emergence of a large number of companies and significant growth of the hyperscalers themselves. I think that's about it. The terms supporting and disruptive technology come from the innovator's dilemma. The idea is that when a new technological trend emerges, there is a power curve. At first, it looks almost like a toy or something that doesn't work well and captures only the lower segment of the market. But as the technology develops, it moves up this curve and eventually begins to disrupt even the positions of existing leaders. At first, leaders don't pay attention to it because it looks like a toy, and then at some point it disrupts everything and absorbs the entire market. This was the case with personal computers. When PCs first appeared, major mainframe manufacturers didn't take them seriously. At first, it was something like a toy for children, and they had large computers, data centers, and so on. And now even data centers run on PCs and their derivatives. That is, PCs became a colossal disruptive technology. But there are also technologies that, on the contrary, greatly help existing players and offer almost no advantage to new companies.
startups. I think Adam is right. Both work here. Perhaps for the first time, we are seeing precisely such a situation. The internet was an extremely disruptive technology, but now it feels like it is simultaneously an obvious amplifier for market leaders, for hyperscalers, large internet companies. And at the same time, it allows for the creation of new business models that are, in some ways, opposed to existing ones. Although, it seems to me, what also happened is that everyone read this book and learned how not to be disrupted. For example, ChatGPT was fundamentally counter-positioned against Google, because Google already had a working business. ChatGPT was perceived as a technology that hallucinates a lot and generates unreliable information. And Google always strove to be a source of trust. Google had its own ChatGPT internally. But they didn't release it for almost 2 years after ChatGPT appeared. And during that time, it played a role in brand recognition, at least. In a sense, OpenAI was released as a disruptive technology, but now Google has realized that it is a disruptive technology and has started to respond to it. At the same time, it has always been obvious that Google benefits in any case. At least their search result answers have become significantly better. The workspace in principle is becoming better with Gemini. Mobile phones, everything is becoming better. So it creates the feeling that it is both at the same time. I strongly agree with that. It feels like everyone has read this book. And this changes the very meaning of the theory, because now all investors in public markets have read it and will punish companies for lack of adaptation and reward them for adaptation, even if it requires long-term investments. I think all company management leadership has also read this book and is in full combat readiness. Plus, it seems to me that the people who manage these companies now are, on average, smarter than the company leaders of the generation on which this book was actually written. They are at the peak of their form. Many companies are controlled by founders, and therefore it is easier for them to endure short-term losses and make such investments. So, to be honest, I think that if we were in an environment like the one in the nineties, this technology would be much more disruptive than in the hyper-competitive world we live in now. One mistake that we, as Verm, have been reflecting on in recent years, although of course I've only been here a few months, is the idea that we refused to invest in companies because they were not supposed to become market leaders or category winners. And we thought: "Well, learning from the Web 2.0 era, you need to invest precisely in the category winner. That's where consolidation will eventually happen and the main value will accumulate." And therefore, it seemed, why create another company with a fundamental model if the first one already has a head start? But now the market has become so much larger in both fundamental models and applications that several winners emerge at once. They sort of fragment the market and take parts of it. And all these parts have venture scale. I'm interested to know if this is a sustainable phenomenon or not, but it looks like one of the differences from the Web 2.0 era. More winners in more categories. I think network effects now play a much smaller role than in the Web 2.0 era, and this makes it easier to launch competitors. Of course, the advantage of scale still exists. If you have more users, you get more data. If you have more users, you can attract more capital. But this advantage does not make competition absolutely impossible for smaller companies. It complicates the task, but it definitely leaves more room for several winners than before. I think another difference is that people so clearly see the value that they are willing to pay at an early stage. In the Web 2.0 era, the question often arose: "How will these companies even make money?" About early Facebook, about Google, and so on, it was always there. How will they monetize? Here, companies make money from the very beginning. Yes, and I think that the previous generation of companies' monetization heavily depended on scale. You couldn't build a good advertising business until you reached millions or tens of millions of users. And now, thanks to subscriptions, you can take money immediately, especially thanks to tools like Stripe, which have greatly simplified this process. And this has also made the market much more friendly for new players. There are also questions of geopolitics. For example, it is obvious that we no longer live in an era of globalization, and perhaps the situation will get even worse. Therefore, investing in a fundamental model, a hypothetical European OpenAI, might be a good idea. China is similar – it's a completely separate world. So there's an interesting geopolitical aspect here. And suddenly all our geopolitical, so to speak, geek expertise becomes useful. Adam, you talked about human knowledge. Did you consider Quora as some kind of self-destruction of your own product in a sense? Or tell us about the bet you made by launching Quora and how you saw its evolution. You know, I think we perceived Quora more as an additional opportunity rather than a threat to Quora. How did we arrive at this? In early 2022, we started experimenting with using GPT-3 to generate answers on Quora. We compared them to human answers and realized they weren't as good. But at the same time, there was something truly unique. You could instantly get an answer to any question you wanted to ask. And we realized that this doesn't necessarily have to be public. In fact, people are more likely to want it to be private, and we felt that a new opportunity was emerging here to let people communicate with AI in a personal space. Yes, and it seemed that you were also betting on how different players would develop, that there would be several of them. Yes, it was a bet on the diversity of companies creating models, and it took time for this to start to materialize. But now, it seems to me, we are approaching a point where there are many models, many companies. Especially if you look at different modalities of image, video, audio models, plus reasoning research models. They are starting to diverge, agents are becoming an independent source of diversity. So we were lucky to enter a world where there is now enough diversity for a universal aggregator interface to make sense. But yes, it was indeed an early bet at first. We, in fact, are surprised that even non-technical users use multiple AIs, and I didn't expect that. Previously, people would only use Google. They didn't compare Google with Yahoo or did so very rarely. But now you talk to ordinary people, and they say, "Yes, I mostly use ChatGPT, but Gemini is better for such-and-such questions." So yes, the level of user sophistication has noticeably increased. And people even say that different models have different personalities, and they, for example, resonate more with Claude or someone else. I want to return to what we discussed earlier, Adam, when you talked about dark matter, about brute force. There is a huge amount of knowledge that people possess but which is not yet structured. And it's not just knowledge about performing tasks, it's knowledge that you can ask about, and a person can describe it. One of the main questions about language models. We've already trained them on the entire internet. How much more knowledge exists at all? Is it 10X, 1,000x? What is your intuitive feeling, if we just build a huge machine head-on that extracts all knowledge from people and turns it into a dataset that can then be used? What advantage do we see here? It's very difficult to quantify. But a huge industry is now forming around translating human knowledge into a form suitable for AI. These are companies like SKAI, Search, Merc, but there is also a huge long tail of other companies that are just starting to emerge. And as intelligence becomes cheaper, more powerful, and more accessible, the bottleneck, it seems to me, will increasingly become data. What data is needed to create this intelligence? This will stimulate more such processes. Perhaps people will be able to earn more money by training, and perhaps more such companies will be launched, and perhaps other forms of this process will emerge. But in general, it seems to me that the economy will naturally value more and more what AI cannot yet do. Is there any framework for what AI cannot do? How can this be formalized at all? Honestly, I don't know. If you ask an AI researcher, they might have a clearer answer. But for me, it's quite simple. There is information that is not in the training dataset. And that, by definition, is what AI cannot do. Yes, AI will become very smart. It will be able to perform complex reasoning. It may be able to prove any mathematical theorem if given a system of axioms. But if it doesn't know how a specific company solved a specific problem 20 years ago, and if that wasn't in the training data, then only a person who remembers it can answer that question. Then how do you see Quora's interaction with this world over time? How do you launch all this in parallel? How do you think about it strategically? Yes, Quora remains focused on human knowledge and on enabling people to share their experiences. This knowledge is useful to other people, and it is also useful for AI as training material. We have partnerships with some AI labs, and Quora will play the role it is intended for in this ecosystem: to be a source of human knowledge. At the same time, AI makes Quora significantly better. We have been able to greatly improve the quality of moderation, answer ranking, and the overall user experience. The application of AI has noticeably enhanced the product. I want to talk about your future. Obviously, you had a business that was developer-oriented for a long time. At some point, you focused on the non-profit sector. No, more precisely, on the EdTech market. As far as I remember, you then reported revenue of $2-3 million. And then, relatively recently, TechCrunch, I understand that the data may be outdated, wrote about approximately $150 million. And as far as I know, given the recent growth, this figure is even higher now after changing business models and target segments. How do you see the future of Replit? I think Karpathy recently said that a decade of agents awaits us. And I completely agree with that. If we compare it to previous stages in programming, first there was autocompletion with Copilot, then the chat approach with ChatGPT, then CERS offered the composer mode, where large chunks of code are edited. Well, that's all. What Replit did is an agent. The idea is that it doesn't just edit code, but manages the entire development environment, sets up infrastructure, databases, performs migrations, connects to the cloud, deploys, runs code, runs tests, debugs. The entire development cycle is enclosed within one agent, and this will develop for a long time. Our agent went into beta in September 2024. It was the first product of its kind that worked with both code and infrastructure, but it was quite raw and not very stable. Then Agent V1 came out around December, on a new generation of models. We switched from Claude 3.5 to 3.7. And it was 3.7 that became the first model that could actually work with a computer, with a virtual machine. It's no surprise that it was also the first computer-using model. These things develop in parallel. With each new generation of models, we unlock new possibilities. Agent V1 could work autonomously for about 2 minutes. Agent V2 for about 20 minutes. Agent V3 we advertised as capable of working for 200 minutes, simply because it sounded nice and looked symmetrical. But in fact, it can work almost indefinitely. We have users who have run it for 20+ hours straight. The main idea was that if you add verification to the cycle, everything starts to work differently. I remember reading an Nvidia article about DeepMind using DeepMind to write kernel code. They managed to run DeepMind for about 20 minutes if a verifier was built into the process, such as the ability to run tests or something similar. I thought, "Okay, and what verifier can we build into our cycle?" The obvious option is unit tests, but unit tests don't actually show if the application as a whole works. Therefore, we started to delve deeper into the idea of using a computer. Can the model test the application itself? This approach is very expensive and also quite buggy for now. As Adam said, this is a big growth area, and it will open up many new scenarios. As a result, we have built our own framework with a lot of hacks, elements, and research that simulate computer usage. In my opinion, this is one of the best ways to test models. When we integrated this into the cycle, it became possible to run Replit at a high level of autonomy. We have an autonomy scale. You choose how autonomous the agent will be. Then everything happens automatically. It writes code, runs tests. If an error occurs, it reads the log, rewrites the code, and continues. It can work like this for hours. I've seen people create amazing things just by letting the system run for a long time. Of course, it needs to get better, cheaper, and faster. Working for a long time is not a reason to be proud. Everything should happen as quickly as possible. We are working on it. Agent 4 will introduce many new ideas, but one of the key ones is that you shouldn't just wait for the completion of one task you requested. You should be able to work on multiple features at once. Therefore, the idea of parallel agents is very important to us. You can request a login page, parallel payment processing via Stripe, and then an admin panel. And it should be able to decide for itself which tasks can be performed in parallel and which cannot, while correctly merging the code. The ability for multiple AI agents to collaborate is critically important. This is how the productivity of a single developer increases dramatically. Now, even if you use CodePilot, Coder, and similar tools, there is almost no real parallelism. I think the next leap in productivity will occur when a developer sits in an environment like Replit and manages dozens of agents. Perhaps hundreds over time, but at least 5, 6, 7, 8, 9, 10 agents working simultaneously on different parts of the product. And I also believe that interfaces and user experience are currently underdeveloped. Essentially, you have to translate your ideas into text in the format of a PRD or product description, as product managers do. But product descriptions are really difficult. This is evident in many tech companies. It's very hard to agree on a precise set of features because language is inherently imprecise. Therefore, I think we are heading towards a world where interaction with AI will be much more multimodal. For example, you open a virtual whiteboard, draw, build diagrams with AI, and work with it just like with a human. The next stage is better memory, both within a single project and between different projects. And perhaps different instances of fast agents will appear. For example, one agent is excellent at data Python because it has all the information, skills, and memory about my company and what it has done before. I will have a separate agent for data analytics, a separate one for the frontend, and they will have memory accumulating from project to project over time and through all interactions. Perhaps they will live directly in Slack as full employees that you can simply talk to. I could talk for another 15 minutes about a roadmap designed for three to four or even five years ahead. But in short, the current phase, the phase of agent systems, is just beginning. There is a huge amount of work to be done, and it will be very exciting. Yes, I recently spoke with a mutual acquaintance, the co-founder of a large productivity company. He heads their R&D. And he said, "You know, during the work week, I hardly communicate with people anymore. I just use all these agents to create something." In a sense, life in the future has already become part of the present. There's something curious about that. People in companies are indeed starting to communicate less with each other. And the question is whether this is good or bad. I increasingly think about the second-order consequences. For example, how will this affect graduates who are just entering the profession? I sincerely pity them. If people stop sharing knowledge among themselves, if it becomes culturally awkward to ask for help because it's assumed you should just use AI agents. Serious cultural shifts are emerging here that we still need to deal with. There are many complex cultural factors for Zoomers right now. I'm already wrapping up. It's clear that you are focused on managing your companies, but to stay in the ecosystem context, you also make angel investments. Where are you most interested right now? We've barely talked about robotics. Are you optimistic about robotics in the near future? Or are there any new categories, use cases, areas where you plan to invest? I generally believe that low-code is an incredibly high-potential topic. The very idea that all this... Do you think it's still undervalued? I think so. It seems to me that the very idea of opening up software capabilities to a mass audience, to literally everyone, is already colossal. Honestly, one of the reasons I consider this direction undervalued is that the existing tools are still very far from the level of a professional software engineer. But if we imagine that they reach this level, and I see no reason why this shouldn't happen, yes, it will take a few years, but eventually anyone in the world will be able to create things that previously required a team of 100 professional programmers. This will radically expand opportunities for everyone. Replit is a great example of this approach, but I think similar use cases will emerge far beyond just application development. By the way, since we're on the subject, if you were entering Stanford or Harvard today in 2025, would you still choose computer science or just focus on building something? I think so, I would. I entered college in 2002, right after the dot-com bubble burst. And there was a lot of pessimism then. I remember my roommate's parents telling him, "Don't study computer science, even though he really liked it." And I just went there because I liked it. I think that yes, the job market is worse now than it was a few years ago. But at the same time, understanding the basics, what is possible with algorithms and data structures, is actually very helpful, especially when you are managing agents and using them. I assume this will continue to be a valuable skill. Plus, there's the question: what else is there to study? For almost any field, you can argue that it will be automated. So, I think it makes sense to study what you truly enjoy. And computer science is no worse than other options in this regard. Yes, there's something to get excited about here. There's one, perhaps a bit random, thing, but I'm very inspired by such crazy scientific experiments. For example, Psychosiar, which came out the other day. Did you see it? It's just wild. Correct me if I'm wrong, because I looked at it quite superficially. But the point is that you can work with the context window much more economically if you have a screenshot of the text, rather than the actual damn text. Yes, I'm not the best person to correct you here, but there are definitely very interesting things there. Yes, I also saw a joke on Hacker News the other day about text diffusion, where someone made a text diffusion model. Instead of Gaussian noise, they took one instance of BERT and essentially masked different words and tried to predict tokens. We now have a huge number of components, but I think people don't think about it that often. We have basic pre-trained models. We have reinforcement learning models for reasoning, encoder-decoder architectures, diffusion models. There's a lot of all this. The idea of simply combining them in different ways seems incredibly promising to me. And I would like to see a new research company that doesn't try to directly compete with OpenAI. Instead, it just tries to figure out how to assemble all these different components together to create a new flavor of such models. Yes, in crypto, they talk about composability, about mixing primitives, and perhaps AI also needs more experiments in this vein. It seems to me that there is less play for play now. I remember the Web 2.0 era when we experimented with JavaScript, browser capabilities, Web Workers, and all that. There were many strange, unusual, but very interesting experiments. In fact, Replit grew out of this. The very first open-source version of Replit, even before the company appeared, was born from the question: can C be compiled into JavaScript? It seemed incredibly interesting. In the end, it became possible through Emscripten, and at that time it was a terrible hack. But it seems to me that we are now in such a Silicon Valley era where everything is very much tied to getting rich quickly. And that saddens me a bit. Partly why I moved the company out of San Francisco. I have a feeling that the culture of San Francisco has become, maybe I didn't experience it personally, but during the dot-com era, many said that there was also a get-rich-quick mentality, like with scripting. And it seems to me that there is a serious lack of tinkering, tuning, and experimentation right now. I would like to see more of that and more companies that receive funding simply for trying to do something more novel, even if it doesn't mean creating a fundamentally new model. Last question. Adam, you've been interested in consciousness for a long time? Are you optimistic that thanks to work on AI or some other scientific progress, we will still make progress in understanding this hard problem, at least somewhere? You know, something interesting happened recently. Claude 4.5 seems to have become more aware of its context length. When it approaches the end of the context window, it starts to use tokens much more economically. It also seems that its understanding of when it's being red-teamed or when it's in a test environment has sharply increased. And there's something quite curious happening there. But if we talk about consciousness, it's still a fundamentally unscientific question. We have, in a sense, given up trying to make it scientific. And the same problem I mentioned earlier manifests here. All the energy goes into LLMs, and almost no one is trying to truly think about the nature of intelligence, the nature of consciousness. And there are a huge number of truly basic, fundamental questions there. One of my favorite works is Roger Penrose's book, "The Emperor's New Mind." In it, he wrote about how in the philosophy of consciousness and in the broader scientific community, people began to view the brain as a computer. And in this book, he tried to show that the brain, in principle, cannot be a computer, because people are capable of doing things that Turing machines cannot do or on which they fundamentally get stuck, for example, on simple logical paradoxes that we can recognize but which are impossible to formally encode in a Turing machine. Let's say, the statement - this statement is false. A classic logical paradox. In general, the argument is quite complex, but if you read this book or many other works, then in the theory of consciousness, there is a whole layer of arguments that computers are fundamentally different from human intelligence. So yes, honestly, I haven't updated my views on this topic much. I've been very busy. But it seems to me that there is a huge field for research here that is currently almost unstudied. If you were entering college as a freshman today, would you study philosophy? I would. I would definitely go into the philosophy of consciousness and, most likely, would pursue neuroscience, because, in my opinion, the key questions that will become extremely important as AI increasingly influences us are located there. Great place to end. Thank you for coming to us.