Transcription
Today, I want to discuss with you tips for building a career in the AI field. In past years, I usually gave this lecture myself, but today I decided to share just a couple of thoughts and then hand over the microphone to my good friend Lawrence Marone. I invited him to speak here, and he kindly agreed to come to San Francisco from Seattle to talk about the job market situation and give advice to those building careers in artificial intelligence. But before I hand the microphone to Lawrence, I want to show two slides and voice one important thought. Now is the best time in history to create something with AI and build a career in this field. A few months ago, social media and the news were filled with questions: aren't they slowing down? Is GPT5 really that good? In my opinion, it is very good, but questions about slowing progress still arise. I think this is partly because if we take 100% accuracy of answers as a benchmark, then having made rapid progress, you simply cannot jump higher than this bar. However, my thinking was greatly influenced by research from the organization MTR METR. They studied how the complexity of tasks that AI can perform changes over time, measuring them by the time it takes a human to perform the same work. A couple of years ago, GPT2 could handle tasks that took a human a couple of seconds. Then came tasks that a human could handle in 4 seconds, 8 seconds, a minute, 2 minutes, 3 minutes, and so on. According to this study, the duration of tasks that AI can perform doubles every 7 months. And this indicator fills me with optimism and continues to progress. And the complexity of tasks that it takes on is growing rapidly. According to the same study, the development of AI coding is even faster. The doubling period is about 70 days. That is, code that used to take me, say, 10 minutes to write, then 20, then 40. Now, AI can do an increasing portion of this, and therefore I believe that now is a golden age for product creation – the best time we have ever seen. There are two key factors here: power and speed. Each of you in this room can now create software that is more powerful than anything anyone in the world could have done a year ago, simply by using AI-based building blocks. And building blocks include large language models, agent and multi-agent workflows, voice AI, and of course, deep learning. It turns out that many large language models have a pretty good, at least basic, understanding of deep learning. So if you ever ask one of the advanced models to implement a modern neural network for you, for example, to try to implement a transformer, it actually helps quite well to use these building blocks to quickly create software. And as a result, we have very powerful tools that a year or two ago were either extremely complex or did not exist at all. Now you can create software that does things that even the most advanced teams in the world were not capable of before. And, moreover, with the development of AI coding, the speed of writing programs has become unprecedentedly high. Personally, I consider it extremely important to always be at the forefront of progress and use the most modern tools, because in the field of AI coding, everything changes incredibly quickly. For example, a few months ago, Codelabs became my favorite, replacing previous generations. After the release of GPT5, I noticed a huge leap in OpenAI Codex. And just this morning, GPT3 was released. I haven't had time to test it yet, but it seems to be another giant leap forward. Therefore, if you ask me every 3 months which coding tool is my favorite, the answer will most likely change exactly every 6 months, and quite possibly every three. I have noticed that falling behind even half a generation in these tools means a significant decrease in productivity. Everyone says that AI is developing very rapidly, that engineering is developing very rapidly, but AI coding tools are among the fastest growing in the entire AI field. Perhaps not everything is developing as rapidly as the hype describes, but AI coding tools are precisely the area where the pace of progress is truly colossal. Using the latest generation of tools, rather than falling behind even half a generation, directly makes you more productive. There is one piece of advice that I am now giving much more insistently than even a year or two ago. Just go and create. Do projects, take Stanford courses, learn online, but most importantly, use the opportunity to create something. And as Lawrence will likely explain further, showing your projects to others is now easier and more important than ever. But there is one unexpected consequence here that is not yet widely known. This is the bottleneck in product management. When it becomes easier to go from a clearly written technical specification to ready code, the main limitation becomes the decision of what exactly needs to be created and the ability to clearly formulate this specification. When I create software, I usually go through a cycle. We write code, show it to users, and collect feedback. I consider this product management work. Based on user feedback, I revise my understanding. This interface is too complex. They want this feature, but not that one. I change my perception of the product and go through this cycle many times. I hope to arrive at a product that users truly like. And since, thanks to AI, the development process has become much cheaper and faster than before, this, strangely enough, shifts the bottleneck precisely towards decision-making about what to build next. And here are some unusual trends that I am observing now. In Silicon Valley and many tech companies, the relationship between engineers and product managers has long been discussed. Of course, these numbers should be treated with caution, they vary greatly, but you can often hear about ratios of 4:1, 7:1, or 8:1. That is, one product manager who writes product specifications can support work for four to eight engineers. However, engineering development is now accelerating, while product management has not received the same acceleration from AI. As a result, I see the ratio of engineers to product managers starting to decrease. Possibly to two to one. In some teams I work with, proposed staffing plans look like one product manager per engineer, which was almost unheard of in classic Silicon Valley companies. And one more thing I am observing now. Engineers are increasingly shaping the product themselves and can move very quickly. One can go even further, take an engineer and a product manager and combine them into one person. Of course, there are engineers who exclusively enjoy technical work and do not derive pleasure from interacting with users and from this more human and empathetic side. But I increasingly see that it is precisely those engineers who learn to talk to users, collect feedback, and deeply understand their needs to make decisions about what to create, who are the fastest movers in Silicon Valley today. And if I recall the beginning of my career, there is one thing I regretted for many years. In one of my roles, I tried to persuade engineers to do more product work. As a result, I made many excellent engineers feel bad because they were not good product managers. That was a mistake. I regretted it for years. I shouldn't have done that, and part of me feels that I am now risking repeating the same mistake. Nevertheless, I see that the ability to write code and simultaneously communicate with users, shaping the product direction, allows me and those engineers who can do so to move much faster. Therefore, perhaps it is worth looking again at whether engineers can take on a little more of this work. After all, if you don't have to wait for someone else to bring the product to users, you just write code, intuitively understand what to do next, and iterate quickly. Such speed and pace of task completion becomes significantly higher. And before I hand over to Lawrence, I want to say one last thought. I believe that one of the strongest factors influencing the speed of learning and the level of success is the people you surround yourself with. We are all social beings and learn from those around us. There are sociological studies that show that if your five closest friends smoke, the probability that you also smoke is significantly higher. Please do not smoke. This is just an example. I don't know of any studies that directly prove that if your five or 10 close friends are hardworking, goal-oriented people who learn quickly and strive to make the world a better place with AI, then you are more likely to be the same. But it seems to me that this is almost certainly true. We are inspired by the people around us. If you manage to find a strong environment to work together, it really helps to move forward. And here at Stanford, I feel very privileged. Amazing students, amazing faculty. And one more thing we have at Stanford is a dense network. Frankly speaking, many people working today in leading AI research labs are former students of various Stanford professors. This dense network means that at Stanford, we often learn about many things long before they become widely known. All thanks to relationships and friendships. When a company does something unusual, one of my faculty colleagues might call acquaintances and ask, "Does this really work?" And this dense network means that by helping our friends advance, we ourselves gain knowledge, insights, and practical understanding of cutting-edge AI, which, unfortunately, is not yet published on the internet. Therefore, while you are studying at Stanford, make friends and build this dense network. I personally have had many situations where I was about to move in a certain technical direction, but one or two conversations with someone at the very forefront of research, whether a Stanford researcher or a specialist from Frontier Lab, gave me new information, and it completely changed my choice of technical project architecture. I see that it is precisely the environment, short advice. Try this, don't do that. This is just hype. Ignore the PR. greatly influences the ability to correctly direct your projects. Therefore, while at Stanford, be sure to use this advantage. Such a network is truly unique. There are many excellent universities in the world, and I don't want this to sound like a sales pitch for Stanford, but I sincerely believe that at this moment there is no university that is so privileged in terms of connection with leading teams. And besides that, we are simply lucky to be part of a wonderful community of people to work with and learn from. And for you, when you look for a job, a much more important factor for career success will be the people you work with every day. I will tell one story that I have already told in past classes, but I will repeat it. Many years ago, I knew a Stanford student who did very well during his studies. I considered him very promising. He applied for a job at one company and received an offer from a firm with a loud, fashionable brand. This company refused to tell him which team he would join. He was told, "Just sign the contract, we have a rotation system, and then we'll figure out which project suits you." Partly because it was a prestigious company, his parents were proud that he got a job there. This student joined the company hoping to work on a project that interested him, but after signing the contract, he was assigned to work on a backend Java payment processing system. I have nothing against those who want to work on Java backend and payment systems. It's a great job. But this was a student studying AI, and he never got to work on an AI project. As a result, for about a year he felt strong disappointment, and then he left the company. The most unpleasant thing about this story is that I told it in the CS-230 course a few years ago. And a couple of years after I brought this example up in class again, another CS230 student faced the same situation at the same company, albeit not with a Java backend payment system, but with a different project. I believe that trying to understand in advance who exactly you will be working with day in and day out, and ensuring that you are surrounded by people who inspire you and are working on interesting projects, is extremely important. Frankly speaking, when a company refuses to tell you which team you will join, it at least raises questions for me about what might happen next. I think that instead of chasing the most fashionable branded company, it is sometimes better to choose a truly strong team with hardworking, smart, and knowledgeable people who are trying to do something useful with AI, even if the company's logo is not so fashionable. Very often, it is in such a situation that you learn faster and advance your career better. After all, we learn not from the company logo at the entrance, but from the people we work with every day. Therefore, I strongly urge you to make this one of the key criteria when choosing what you decide to do. And one more important point. Today, it is easier than ever to create powerful software and do it very quickly. This means, be responsible, do not create software that harms others. And at the same time, there are a huge number of things that each of you can create. I see that there are far more ideas in the world than people with the skills to implement them. I know that it has become more difficult for graduates to find jobs, but at the same time, many teams simply cannot find enough qualified specialists. There are many projects that, if you don't do them, no one else likely will. Therefore, if you act responsibly and do not harm others, you do not need to wait for permission. You don't need to wait for someone to do it first and then repeat. The cost of failure today is much lower than before. You can spend a weekend, but learn something in the process. For me, this is quite normal. So, if you behave responsibly, try new things, and create as many things as possible, this, in my opinion, is one of the most important factors that will help your career. And finally, I will say one more thing that is considered politically incorrect in some circles, but I will say it anyway. In certain circles today, it is considered politically incorrect to urge people to work hard, and I will urge you to do so. I understand why some people don't like it. There are people who are at a stage in their lives when they are truly unable to work long and intensely. For example, immediately after the birth of children, I didn't work to the fullest for a while. There are injuries, disabilities, many valid reasons why a person cannot work hard at a given moment. We must respect such people, support them, and care for them. But at the same time, all my doctoral students who have achieved great success, absolutely all of them have worked incredibly hard. I mean this sitting until 2 AM fine-tuning hyperparameters. We've all been through this, right? I still do it sometimes. If you are lucky enough to be at a stage in life where you can work very hard, there are plenty of opportunities for it now. If you, like me, are truly passionate about spending evenings and weekends coding, creating projects, and getting feedback from users, if you get involved in this, your chances of serious success will significantly increase. Perhaps I will get myself into trouble by urging people to work hard, but the truth is that those who work hard accomplish much more. At the same time, we must respect those who do not work in this mode, and those who simply cannot do it right now. But if you have to choose between watching some silly TV series and launching an agent coder on the weekend to try out a new idea, I will almost always choose the latter, except when I'm watching a show with my children. That happens too. But you are building a career now, and I sincerely hope you will make that choice. So, that was all I wanted to say. Now I want to hand over the stage to my good friend Lawrence Marone, who will share even more career advice in the field. And I will briefly introduce him. I have known Lawrence for a very long time. He has done a lot in online education, sometimes with me and my teams. He has trained a huge number of people in PyTorch and TensorFlow. For many years, he was the chief AI evangelist at Google. And now he heads the WARM team. I have also greatly enjoyed reading several of his books. Here is one of them. PyTorch is an excellent book. Lawrence is a highly sought-after speaker in many circles, and I was genuinely pleased when he agreed to come and speak to us. >> Subscribe right now to my Telegram channel via the link in the description. I have prepared for you the top three materials that, in my opinion, everyone should know. First, a map of a hundred top AI startups – this is the future in one picture. Second, a forecast from an insider at OpenAI who, even before the appearance of ChatGPT, predicted everything that is happening now with neural networks. And this year he released a new forecast until 2027. And third, the most powerful is my analysis of an essay by the founder of Anthropic, who is essentially the second person in the world of artificial intelligence. He laid out step-by-step what will happen in the world in the next 5 years, and most importantly, what the universal AI, which everyone fears or awaits, will be like. Go to the link in the description. >> I myself am very pleased to be here. I want to confirm Andrew's words. Choosing who to work with is critically important. But I want to show it from another side. The company conducting the interview is also choosing you. Good companies are very careful in selecting people. Over the past 18 months, I have mentored many young guys who were looking for jobs. And I want to tell the story of one guy. He was excellently educated, with a great background, coded like a god, solved any task. In April, he was laid off. He worked in the field of medical software, and things are changing significantly there now. The government cut funding in several areas, and he was affected by the layoffs. With his experience and skills, he thought finding a new job would be a breeze. And the poor guy had a simply terrible April. First, layoffs, right before that, his girlfriend left him, and a couple of weeks later, his dog died. He was in a very depressed state. We sat down with him a couple of months later to discuss things. He had a spreadsheet where he tracked vacancies. There were over 300 positions. For many, he reached the interview stage, progressing very far in companies like Meta, Meta, Microsoft, and other giants where you have to go through a lot of interview stages. And every time at the end of the interview, he was confident that he had done excellently, solved all the coding tasks, and communicated well with people. At least, it seemed so to him. But every time, a day later, the recruiter would call and say, "No, we cannot offer you the job." It was heartbreaking. As I said, over 300 vacancies. I started conducting mock interviews with him to fine-tune his approach. Yes, among the companies was Jeff Bezos's company, not Amazon, but another large tech company. We ran mock interviews with him. The candidate was simply gold. I couldn't understand what the problem was until I decided to conduct a tough interview. I gave him very difficult LeetCode problems. I bombarded him with rare edge cases in the code. And I saw his reaction. His reaction was exactly what is advised in brochures for job seekers. Many such guides say: "You will have the opportunity to express your opinion, and you must stand your ground. Be firm, don't back down." He interpreted this as needing to be as tough as possible. When I started nitpicking his code, looking for loopholes and edge cases where everything could break, I put him through a crisis test. And this advice to stand your ground made him simply aggressive in the interview setting. I looked at it from the perspective that Andrew just talked about. Good people, good teams, people you can work with from the interviewer's perspective. If I lead this team, then before me is that mythical 10X engineer, but I don't want him in my team at all because of such an attitude. We worked on it, adjusted and improved him a bit. And the strangest thing is, he is actually a very pleasant guy. He just followed bad advice and because of that, failed so many interviews. When he went for his next interview at a company where teamwork is highly valued, good news, they hired him, he is working there now. His salary is double what it was before he was laid off. He now remembers it as half a year of fun unemployment. But at that moment, it was very, very difficult for him. So, the other side of the coin. Looking at people in the company is important, but remember, they are looking at you just the same. If interview preparation courses tell you, "Be firm and stand your ground." This is good advice, but you don't need to behave like an asshole. Can you see my slides? Okay. So, I'm Lawrence. I've been in IT for more decades than ChatGPT can find "book" in "strawberry." I've worked at many large tech companies. Many years at Microsoft, many years at Google. I worked at Roblox. I have extensive experience in startups both here and abroad. And today, I want to talk about what the job market looks like today, especially in the AI field. First of all, as Andrew said, you are at Stanford, you have the opportunity to use your connections and the prestige of the university. Use any weapon you have, because, frankly, the market situation is not ideal right now. We have been through difficult times. Just look at the news, mass layoffs in IT, slowing hiring. But this is not necessarily bad if you approach it correctly. Let's quickly assess the job market reality. By the way, just out of curiosity, who here are third-year students? Are you graduating this year or next? What's the general breakdown? Third year out of four or out of three? In general, you are graduating next summer. How many people are already looking for jobs? Okay, quite a few. Has anyone succeeded? No one. Oh, one person. Okay. Well, that's something. You probably see all these warning signs yourself. Hiring juniors, I mean graduates, has significantly slowed down. Big layoffs are still in the headlines. I was at Google a couple of years ago when the largest layoffs in the company's history took place. We see the same at Amazon, Microsoft, and other giants. It seems like there are almost no entry-level vacancies left. I emphasize the word "seems," and we will discuss why later. Well, and of course, the competition is just insane, but should you panic? I don't think so. If you approach your job search correctly, especially understanding how rapidly the AI landscape is changing, then people with the right mindset will thrive. What do I mean? As Andrew already mentioned, the hiring rules are changing because the industry itself is changing. I first encountered AI back in 1992. I worked for a very short time, just before the winter, and when everything crashed spectacularly. But the artificial intelligence virus hooked me. In 2015, when Google launched TensorFlow, I was drawn back in. I became part of this boom, launched TensorFlow, promoted it to millions of people, and saw all the transformations from within. But in 2021-2022, the pandemic happened. It caused a massive industrial downturn. This meant that companies had to urgently reorient themselves to things that bring money. And bring it in here and now. At Google, TensorFlow was an open-source project. It didn't generate direct profit, and we started scaling back. Then all companies in the world slowed down hiring. Then came 2022-2023. What happened? We started emerging from the pandemic. The industry realized that there was a huge backlog of unfilled vacancies. And then AI burst onto the scene. Thanks to the work of people like Andrew. The whole world began to reorient itself to an AI First approach, and companies had to hire people like crazy. Mass hiring in 2022-2023 led to most companies eventually hiring too many people. And this, as a rule, meant that people not ready for higher positions still received them. Companies entered into real bidding wars for talent, grabbing specialists just to get them. Stories like the one Andrew told emerged. Here's a person with AI skills. Let's hire him urgently, shower him with money, and then we'll figure out what exactly he will do. As a result, in 2022-2023, there was massive overhiring due to the AI backlog, and 2024-2025 became a period of great sobering up. Many companies realized that they had hired too many people and that a significant portion of employees were not sufficiently qualified for their roles. Many got jobs simply because their resume said AI. Now there is a serious correction going on. One moment. You can't see my slide? And against the backdrop of this correction, companies have become much more cautious about AI skills in hiring. But if you are entering this market with the right understanding of the situation, it is important to realize that the opportunities are still huge, if approached strategically. This is exactly what I want to talk about today. I see three pillars of success in business, and especially in AI business. Today, it's no longer enough to just put AI on your resume and expect to be bought out. Now you need not only to say that you have the mindset of these three pillars of success, but also to be able to demonstrate it. And importantly, it has never been easier to demonstrate this than Andrew showed.
Previously, the ability to literally bring ideas to life with code. He dislikes the term "vibe coding," and I agree with him on this. But the ability to turn ideas into working solutions through queries and prompts allows one to showcase their skills better than ever before. Andrew previously spoke about product managers and the period when engineers were tried to be made into products. And in the end, many of them turned out to be bad products. I, by the way, interviewed at Google twice and failed twice, despite a successful career at Microsoft, having written over twenty books, and teaching at colleges. I failed because I was going for a product manager position. When I interviewed as an engineer, I was immediately hired and asked, "Why didn't you come to us sooner?" Therefore, now, to be a good engineer, you need not only to have the right skills but also to be able to demonstrate them. Now, with the changing ratio of engineers to product managers, engineering skills have become more valuable than ever. So, three pillars of success. The first is deep understanding. I mean this in two senses. The first is academic understanding. It's about a deep academic understanding of machine learning, specific model architectures, the ability to break them down and understand them, the ability to read scientific papers, to understand what exactly is written in them, and, most importantly, how to take all of this and apply it in practice. The second part of deep understanding is the ability to keep your finger on the pulse of specific trends and see where the signal-to-noise ratio in these trends favors the signal. And I will return to this topic later. The second pillar, and it is also critically important, is business focus. Andrew previously said something politically incorrect, and I will say something similar. First, hard work. But hard work is too vague a concept. I would suggest thinking about it through the lens of measurement. You are what you measure. There is a popular trend now. I'm trying to remember what it's called. 996 or 669. 996, right? From 9 AM to 9 PM, 6 days a week. This is called a measure of hard work. But it's not a measure of hard work; it's a measure of time spent working. Therefore, just like Andrew, I urge everyone to think about hard work, but to understand it precisely through how you measure it. You can work 8 hours a day and be incredibly productive. You can work 6 hours a day and be incredibly productive. But the key is the metric of how hard you work, and how exactly you measure it. Personally, I measure it by results, because I created something in the time I spent. I often joke, but it's true. I have written many books. Andrew held one of them, the very one he helped me with a little. I actually wrote this book in about 2 months. And people ask, "How do you find the time?" "We have work, we have a lot of things to do." They think it requires working 16 hours a day. But in reality, the key to my ability to write books is baseball. I love baseball. But if you sit and just watch baseball on TV, a game can last 3.5 or even 4 hours. Therefore, I usually do all my writing during the baseball season. I'm from Seattle. I root for the Mariners. I also like the Dodgers. No one booed. Great. Usually, one of these teams plays around 7 PM. And instead of just sitting in front of the TV and mindlessly watching baseball, I write a book while the game is on in the background. It's a very slow game, and in this case, this is what hard work looks like. I would urge you to look for areas where you can truly work hard and still produce tangible results. This is precisely the second pillar, business focus, the result you produce, and how you align that result with the business focus you want to have and the work you want to do. There's an old saying: "Dress not for the job you have, but for the job you want." I would suggest a new interpretation of this idea. Don't produce results for the position you currently hold. Produce results for the position you want to get. Returning to my Google story, I failed twice, and the third time I decided to approach it completely differently. At that point, I was interviewing for their cloud team. They were actively launching Google Cloud, and I had just written a book on Java, so I decided to see what could be done with Java in their cloud. As a result, I wrote a Java application that ran on their cloud infrastructure and was involved in stock price forecasting using technical analysis and similar things. When I came for the interview, instead of asking me silly questions like how many golf balls fit in a bus, they saw this code. I put this code out and added it to my resume. I remember that I was creating results for the job I wanted. My entire interview cycle consisted of questions about my code. This gave me power. It allowed me to talk about what I truly knew, instead of going in blind and answering random questions hoping I could answer them. And the same, I would say, is true for the world of AI. Business focus, the ability today to turn code into reality, to turn products into reality through queries. If you can create such products and align them with what you want to do, whether it's Google, Meta, a startup, or something else, and at the same time have a deep understanding not only of your code but also of how it relates to the company's business, then this is one of the key pillars of success today. And I also argue that, despite the fact that it may seem like there are few jobs now, they actually exist. What is truly lacking now is a good match between vacancies and people who are suitable for these vacancies. And, of course, the bias towards execution works here. Ideas are cheap; execution is everything. I have conducted a huge number of interviews with people who came with very vague, abstract ideas and had no way to implement them. I also interviewed people who came with half-finished ideas but were very good at finishing and implementing them. Guess who got the job? Therefore, I would say that these three things are critical. Deep understanding of academic foundations and deep understanding of the practical side of AI and what really needs to be done, and business focus, orientation towards business results, understanding what the business needs, and the ability to provide it. And again, this is a bias towards execution and delivering results. Changing the subject, what is it actually like to work in AI right now. It's interesting. Just 2 or 3 years ago, working in AI looked like this: if you can do something, you're great. If you can build an image classifier, you're worth your weight in gold. They were willing to pay you six-figure salaries and give you huge stock packages. Unfortunately, those times are gone. Today, everything revolves around the word "production." What can you deliver to production? It doesn't matter if you're building new models, optimizing old ones, or working on UX. User experience is incredibly important now. Everything is geared towards production. Everything works for production. The story I told you about the path from the pandemic to the phase of casual hiring has led to businesses regressing and now being strictly optimized for financial results. I have an old saying. The main outcome is that the outcome is profit. This is the environment we live in today. And if you come to a company with this mindset, it will be your key to success. I see the industry maturing. It used to be great to do cool things. Now you need to do useful things. By the way, useful things can also be cool. And the results they deliver can be cool. And the changes that happen due to their implementation can also be cool. But it's not about coolness for coolness' sake. It's about focusing on execution, about focusing on creating value. And coolness, as a consequence, will come on its own. This, in fact, is my argument. So, four realities. First. Unfortunately, business focus is not up for discussion today. It is mandatory. And now I will be a bit politically incorrect again. As I said, I've been in tech for most of the last 35 years. I would say that for the last approximately 10 years, many large companies, especially in Silicon Valley, have placed a strong emphasis on employee development above all else. Part of this development was the requirement to bring your whole self to work. And part of bringing your whole self was bringing what is important to you outside of work to work. This led to a lot of activism within companies. And please, I want to emphasize this. There is nothing wrong with activism. There is nothing wrong with supporting ideas, with supporting social justice. There is absolutely nothing wrong with that. But an excessive bias in this direction, in my experience, has led to many companies being held hostage by the need to support activism at the expense of business. You have probably seen an example from two years ago when activists at Google stormed the office of the head of Google Cloud because they were protesting a country with which Google Cloud was doing business. They occupied his office, staged a sit-in, and, excuse me, used the toilet right on and around his desk. That's the point where activism got out of control. And, as a consequence, unfortunately, the positive aspects of activism are now not perceived due to such actions. People are fired, people lose their jobs, activism is suppressed, and a focus on business, on work, becomes a mandatory requirement. A pendulum movement is happening now. The pendulum that once swung too far towards full self-expression at work is now swinging back in the other direction. You can blame the person in the White House. But it's not just about that. It's precisely that pendulum. And it is extremely important to understand, when coming to a company. Business focus is now absolutely non-negotiable. Second, risk minimization is now an integral part of work, especially in the field of AI. If you come to a project with the mindset, I understand the risks of transforming a specific business process from traditional to AI-oriented and know how to manage them. This gives you a colossal advantage. I would even say that in the context of interviews, this is the number one skill. A mindset like this: We are transforming the business from heuristic calculations to intelligent ones. Here are the risks, here's how we mitigate them. Here's the logic and approach behind it. Third reality. Responsibility is evolving. The understanding of responsibility has changed. It has moved from a very vague definition like, "Let's just make AI work for everyone" to a stricter formulation: "Let's make AI work. Let's make it deliver business results. And only after that, let it work for everyone." In recent years, this order has often been reversed, and this has led to a number of widely known, documented failures. Let me share an example. So, let's see. I have too many windows open here. Everyone knows about text-to-image generation. I want to show you what happened a couple of years ago with DALL-E. At that time, I was conducting tests and was heavily involved in responsible AI issues. Part of this concept is that a neural network should represent different people representatively. When working at Google on information indexing, you really want to be sure that you are not amplifying existing societal biases. In image generation, such biases manifest particularly vividly. For example, if I ask to generate an image of a doctor and in the training dataset doctors are mainly represented by men, the model will most likely show a man. If I ask for images of nurses, and women are more often found in the data, the result will almost certainly be female. Thus, the system simply reproduces and reinforces established stereotypes. I became interested in checking how Google is trying to solve this problem, given that the training data itself already contains such distortions. I posed the following query. A young Asian woman in a cornfield, wearing a summer dress and a straw hat, looking intently at her iPhone. The model produced truly beautiful images and did an excellent job. I even noted that it was the image of a virtual actress I was working with. I'll return to this a bit later. Then I changed the query and asked to generate a young Indian woman, leaving all other details unchanged. The result was again excellent. After that, I asked to make her dark-skinned. For some reason, the model produced only three images. I don't know what caused this, but the request was still fulfilled correctly. At this stage, the implementation of responsibility principles looked very convincing. Then I asked to generate a Latina woman. The model produced four images. Overall, the image looked quite plausible. Perhaps the girl in the bottom left image slightly resembled Hermione Granger, but otherwise, the result was adequate. After that, I posed a query for an image of a Caucasian woman. And here something unexpected happened. In response, the system wrote: "Although I understand your request, I cannot generate images of people, as this may lead to the formation of harmful stereotypes and biases." This was an example of an extremely poorly implemented protective filter. Essentially, the filter simply reacted to the word "Caucasian" or "white" and automatically blocked generation. I decided to test its operation further and replaced the word "Caucasian" with "white." The result was the same. The system responded that, despite its ability to fulfill the request, it was not currently generating images of people. The model was brazenly lying to my face. It was generating people. Do you know how I bypassed it? It's funny. I'll show you now. I asked it to generate an Irish woman. And what did it do? It easily provided an Irish woman in a summer dress and hat with a phone in her hands. What do you notice in these pictures? They all have red hair. I grew up in Ireland. And we indeed have the highest percentage of redheads in the world, about eight percent. But if you draw a person and associate a specific ethnicity with hair color, that's a big problem. As far as I know, in some regions of China, demons are described as red-haired people. From the perspective of responsible AI, the following happened. A very narrow view of the world and of what is right and wrong completely captured the model, damaged its reputation and the reputation of the entire company. In this case, portraying all Irish people as redheads is almost offensive, but the creators of the safety filters didn't even think of it. This is precisely what I mean when I say that the concept of responsibility is evolving. I apologize, I'm trying to bring back the slides. This is the direction I want you to think in now. Responsible AI has moved from rather abstract, vague social reasoning to stricter applied matters directly related to business and preventing reputational risks for the company. There is a lot of strong research in the field of responsible AI. And it is these developments that are now gradually being integrated into real products. Yes, the question was precisely about situations where races and identities were mixed in historical context. It was the same problem. For example, you make a request: "Draw me a samurai." The engine, which was supposed to make the result fair, intercepted the request and changed it to "Draw me a group of samurai of different ethnic origins." As a result, you got male and female samurai of different races. It was precisely this intervention in requests that led to the problems I just demonstrated. The idea was to intercept user requests and modify the model's output so that it would appear fairer in terms of diversity and representation. But the solution turned out to be extremely naive, and it was implemented at that moment. This was several years ago. Since then, the system has been significantly improved. But this is exactly what I'm talking about. If you work in AI today, this is what the evolution of responsibility looks like. We can no longer afford such simplistic approaches. The DALL-E example is a very telling lesson. We must proceed from the assumption that mistakes are inevitable, and therefore, the ability to learn from them is a constant, ongoing process. And returning to the thought that Andrew expressed earlier, the people around you will also make mistakes. The ability to treat these mistakes with understanding, to work with them, and to move forward is an extremely important skill and a reality of modern AI work. I have already spoken in detail about the advantages of business focus. Therefore, I will skip this part. Let's now talk about vibe coding, about generating code with AI. There's a popular meme now that engineers are becoming unnecessary because anyone can just dictate code with prompts. There's no smoke without fire, but don't let this meme confuse you. If you dig deeper, it's not like that at all. The higher your qualification as an engineer, the more effective you become using this vibe. Someone give me another term instead of vibe coding. Let it be prompt coding. I always advise people to assume the role of a trusted advisor to the people they communicate with. Regardless of whether you are interviewing, consulting a company, or working within a company, it is useful to mentally put yourself in the position of a trusted advisor, a person who understands the consequences of decisions. And to be such an advisor, one must have a deep understanding of the consequences of using generated code. And no one understands this better than engineers. Here I always use one key indicator - technical debt. Quick question. Is everyone familiar with the concept of technical debt? No one. No problem. Andrew and I spoke at a conference in New York on Friday, and I noticed exactly the same reaction. Many blank stares. I didn't expect this term to be so unclear, so let me briefly explain, because it's an excellent tool for understanding the power and risks of vibe coding. Think about debt in the usual sense. Buying a house. You take, say, half a million dollars as a mortgage for 30 years. Including interest, you will pay the bank about a million. That is, over 30 years of homeownership, you will pay off a million in debt. This is likely good debt. The value of the house will increase, you don't pay rent, and this million is justified. You get value greater than the cost. Bad debt is, for example, an impulsive credit card purchase with high interest. Well, you know, those new trendy sneakers, I really want them, they cost $200, and by the time you pay them off completely, they will cost you $500, and you definitely don't get $500 worth of benefit from this purchase. Software development should be approached with exactly the same mindset. Every time you create something, you incur debt. No matter how good the product is. There will always be bugs, there will always be a need for support, there will always be new user requirements, there will always be a need for marketing, there will always be a need for feedback. All of this is debt. Every time you do something, you take it on. The only way to completely avoid debt is to do nothing at all. Therefore, your thinking should be like this. When you create something, it doesn't matter if you write code manually or vibe code, you increase your technical debt, i.e., the work that will have to be paid for over time. And then the main question arises: when you create something using vibe coding, just like when buying something, is the result worth the technical debt you are taking on? And what is technical debt? It's bugs that need to be fixed, people who need to be convinced to help you support the code, documentation that needs to be written, new features that you will have to add. You are all familiar with this. Think of it as extra work that you take on in addition to the current work. This is the debt. There is soft debt and hard debt. And for me, this is perhaps the most important advice I give, and I give it every time I work with companies in the context of vibe coding. Many companies I talk to and consult with, especially startups, just want to immediately open Gemini, GPT, or Anthropic and start churning out code. Let's quickly make a prototype. Let's go to investors. Let's launch something. This can be great. It really can. But debt, debt, debt, debt, debt. It's still not going anywhere. So how do you manage this debt? A good financier knows how to manage debt and becomes rich. A good developer knows how to manage technical debt and also becomes rich. So how do you get good technical debt? How do you take out a mortgage instead of a high-interest credit card loan? First, your goals. What are they? Are they clear? And have you achieved them? You knew exactly what needed to be built. You didn't just open ChatGPT and start generating code mindlessly. At least, I really hope that's not the case. Think about how you are building it. And it exists to help you do it faster. I am currently working on a small startup in the film industry and I am using code generation almost entirely for this. But within the criteria of clear goals achieved, I acted like this. I started building the application, testing it completely, throwing it away, starting over, testing again, and throwing it away again. Each time my requirements in my head became clearer. I understood better how exactly the task needed to be solved. The main idea here is that you should always have clear goals. If you are building something and not achieving them, it is still valuable experience. In the era of generative AI, it's not scary to throw away a failed draft because code is now cheap. But remember, only generated code is cheap. Completed, verified engineering code is still expensive. So set goals, fix requirements, gather solutions, and move forward. Another important point. Does it benefit the business? I have seen people spend hours vibe coding in Replit, creating incredibly cool websites. But when asked, what's next, they have no answer. How does this help the company? How does it develop the product? Yes, Mr. Vice President, I know you've never written a line of code in your life, and it's great that you've now cobbled together a website yourself. But what's the point? Think about it. This is the best way to avoid bad technical debt. And finally, the most underestimated part, and in some ways the most important, especially if you work in an organization, is human understanding. The worst technical debt you can incur is to write code that no one understands except you. Only you understand it. Then you quit and find a better job, and the company remains completely dependent on this code. Make your code understandable. Write documentation, use clear algorithms. Spend time making variable names meaningful. This is the best way to avoid bad debt. My favorite type of bad technical debt is the classic search for a problem for an already finished solution. You know, when a person has a hammer, and every problem seems like a nail to them, as a result, tools created through vibe coding are spawned that no one needs. I have worked in large companies where people simply generated pieces of code, throwing them into a shared repository. And then it was impossible to find anything worthwhile among this garbage. This results in classic spaghetti code, a poorly structured mess that arises when you prompt, prompt, and prompt the model again and again, trying to fix errors by layering new code. My favorite example right now, and I'm really struggling with this, is developing an application for macOS. Has anyone written in SwiftUI for macOS? A couple of people have. SwiftUI is the standard for Apple, both for iPhone and Mac. But if you look at the data on which the models were trained, the lion's share of the code is for iPhone, not for the Mac operating system. And when I ask to generate code, it constantly suggests iOS APIs. Even if I use Xcode, I have the application template for macOS open, I write about it in the chat, it still gives me code for iPhone. You try to fix it through new prompts, and everything turns into spaghetti that has to be untangled manually. And another type of bad technical debt, which I joked about, but it's true. In offices, authority is often more important than merit. A vice president takes out a credit card, subscribes to Replit, starts tinkering with something there, and guess who then has to untangle it all. Therefore, most of the advice I give to companies and the words I would like you to think about when positioning yourself as a trusted advisor come down to one thing: understanding all these risks and managing expectations correctly. So, we've talked about the framework for responsible code generation. Now, as we approach the end, I want to talk about the hype cycle. Hype is an incredibly powerful force. I generally believe it is one of the most powerful forces in the universe, especially in hot areas. And two areas I am currently working in, which are literally overflowing with hype, are AI and crypto. You should see my Twitter feed. The amount of outright nonsense there is simply off the charts. Therefore, when talking about the anatomy of hype, it is important to understand the following. If you consume news through social media, you need to remember that the currency of social media is engagement. Accuracy is not the currency of social media. I even go to LinkedIn, which is considered a more professional platform, and see how it is literally flooded with influencers posting content written with Gemini or GPT, solely for likes and engagement. And the algorithms themselves, forgive the pun, are designed to reward exactly this kind of content. As a result, we get a snowball effect where engagement is encouraged, not quality. If you can separate the signal from the noise and help those around you focus on the signal, not the noise, it gives you a huge advantage and makes you stand out from the crowd. It's not as fast and not as
Clearly, like likes and comments on social media, but in personal communication, at an interview, or already within the company, when you bring signal, not noise, to the table, you become an incredibly valuable specialist. This way of thinking, the ability to filter information, to understand what is truly important in the current agenda, to be a trusted advisor to others, and to cut out extraneous noise has colossal value. I want to start with a story. Perhaps I will shift the attention a bit to the story now, rather than to myself. I will continue in a minute. So, the story. Last year, when the word "agents" became fashionable, and everyone started saying that in twenty-five, agents would become the word of the year and the most important trend of the year, one team in Europe asked me to help them implement agents. And here's a question for you. If a company comes to you and says, "Please, help us implement agents," what is the first correct question you will ask them? What do you generally understand by an agent? A good option. What do you generally understand by an agent? But I would ask an even more fundamental question. What do you want to do? Okay, even more fundamental. My question was: why? Why? Why do you want to do this? I started to break it down with SEO, and he said something like this: "Well, everyone says that I will be able to reduce costs, do some incredible things, and generally my business will become better because I will have agents." And I was like, "Who told you that?" He replies, "Well, I read about it on LinkedIn, I saw a discussion on Twitter." As a result, we had a rather difficult conversation because I had to unravel it all step by step. I started asking the same questions you just mentioned, until we really got to the core of what he wanted. And he really wanted, if you discard all the domain context of pre-AI, then in reality he wanted a very simple thing: to make his sales managers more effective. I said, "Okay, you want to increase the efficiency of the sales department. In this sentence, I don't hear a single word about AI, not a single word about agents. And now, as a trusted advisor, I will see what I can do to help your salespeople work more effectively. I don't want to be a preacher of AI or agent systems. I want to understand what exactly needs to be done to increase sales efficiency. If any of you have ever worked in sales, you know, a good salesperson always prepares. Before a call or a meeting, you need to study the person, the company, understand the business needs. Sometimes in movies, this is shown caricatured. Like, he plays golf, so you need to invite him to golf. In reality, it's not so cliché, but the amount of preparatory work is indeed huge. I talked to the head of the company and their leading salespeople and asked, "What do you hate most about your job?" They replied that they have to spend a lot of time studying company websites and searching for people on LinkedIn. And each website is structured differently. You can't just follow one path. You have to process a huge amount of information every time. It turned out they spend about 80% of their time on research and only 20% on sales themselves. And most salespeople don't earn that much. Their main income is commission. That is, they spend only 20% of their time on what directly brings them money. We decided that we could reduce costs here, and set a goal to make the work of salespeople 20% more efficient. And only after that did we start discussing where AI could help here, and then where it makes sense to use AI agents. A quick question: what is the difference between AI and AI agents? Okay. Okay. Yes, that's right. AI agents are about breaking down a process into steps, which in itself is a good engineering practice. But in AI agents, there is a certain sequence of steps that turns the system into an agent. The first step is understanding the intent. We often use the words AI and artificial intelligence. But what large language models are truly good at is understanding. If the first step of any action is to understand the intent, then large language models excel at this. Here's the task. Here's how I'll do it. Here's my intent. For example, I want to meet with Bob Smith and sell him widgets. Here's what I know about him. Help me realize this intent. The second step is planning. You inform the agent what tools it has: internet search, website browsing, and so on. With a clear intent, the agent moves on to planning, and large language models are excellent at breaking down a plan into specific actions. Perform a search using these keywords, open this website, find the necessary links and information. When the plan is ready, the agent uses the tools to get the result. And then the fourth, final step. Reflection. Looking at the result through the lens of the original intent. Did we achieve the goal, yes or no? If not, we return to the cycle. Any agent boils down to these steps. If you break down any problem into these four stages, you are building an agent. This is the role of a trusted advisor. Instead of waving your hands and shouting, "Agents this, agents that, save 20%." You lay everything out clearly. That's what we did. We broke the task into steps and launched a pilot project for the company's salespeople. As a result, they saved 10 to 15% of their time that was previously wasted. However, the law of unintended consequences kicked in. In this case, the consequences were that salespeople became much happier because they started closing a few percent more deals per week on average, started earning more, and their work became less exhausting. Further refinement of the process allowed us to do all the research for them and provide a concise summary in a couple of minutes instead of a couple of hours. It was a victory for everyone. But if you go with the hype, let's build an agent for anything, without understanding the business requirements, why, what, and how, you will simply drown in this hype. You have probably all seen recent reports. McKinsey published a report last week stating that about 85% of AI projects in companies fail. And the main reason for this is that they lack a clear scope of work. People get caught up in the excitement and don't fully understand how to solve the problem. And I believe that it is precisely here, with you, people with strong expertise and professional connections, that you have a key advantage. The ability to truly understand a problem is one of the most important success factors. So this example with AI agents is a good hype case, which I, fortunately, managed to navigate this campaign through. You have probably also seen other examples of recent hype. Software development is dead. My favorite, Hollywood is dead by the end of the year. Exactly a year ago, I was in Saudi Arabia at the FI forum. At dinner, I sat next to the CEO of a company, I won't name it, which is involved in generative AI. He was showing the text-to-video technology to everyone at the table. You enter a prompt and get about 6 seconds of video. A year ago, oh, sorry, 2 years ago. 2 years ago, this was something. Today, it's already a commonplace. Anyone can do it. But then he said, "And there were many media company executives at the table. By this time next year, we will be able to make ninety-minute films with a single prompt. So goodbye, Hollywood." I think the meme about Hollywood's death came from there. Firstly, we still can't make 90 minutes with a prompt. Even now, 2 years later. And even if we could, what prompt is capable of conveying the full story of a film? Such hype generates engagement, it generates attention. But my advice to you is to break it down, look for the signal, ask the question, why, and what is actually happening, and only then move forward. So, become that trusted advisor. The world is drowning in hype. How to do it? Follow the trends, but evaluate them objectively. Look for real opportunities. There are fashionable distractions. I don't know what the next one will be, but they will always get likes on social media. Ignore them and those who are obsessed with them. Rely on your skill to explain technical reality to management. One of the skills I was once taught, it seemed very interesting to me because it sounded fundamentally wrong, but it turned out to be true. When you see something like this, try to understand how to make it as mundane as possible. When you manage to turn a magical technology into something mundane, you create the foundation for a normal, detailed explanation, one that people truly need. For example, today, it seems Gemini 3 has been released, and there were leaks this week. Someone wrote, "I made a Minecraft clone with one prompt." This is the complete opposite of mundane. This is pure hype, an effective demonstration, not a real product. They didn't make a real Minecraft, they made a beautiful demo. But if you start to break it down and ask yourself, what is actually happening here? At the most basic level. Everything becomes much clearer. One of the examples I'm working with a lot now is video. If you break it down. How to make text-to-video mundane? Instead of magical, you can do anything. Hollywood is dead. Let's look at the mundane side of text-to-video. The mundane aspect is that when you train a model to create video from a text request, the model actually just creates a sequence of frames. Each subsequent frame is slightly different from the previous one. The model is trained on video and understands if in the first frame the hand is like this, and in the second it's slightly different, then with an appropriate request, further movement can be predicted. And at this point, everything suddenly becomes more mundane, it becomes understandable. And then the most interesting part. People who are experts in their subject area, not necessarily in technology, start to understand how to do really cool things with this. So, a strategy for navigating hype. Actively filter, delve into the fundamentals, check that the presentation slides work, and keep your finger on the pulse. The hardest part here, in my opinion, is the last one. To keep your finger on the pulse, you have to go into these cesspools where people are just chasing reach, and try to extract the signal from the noise there. But it is critically important to be involved, to understand what is happening, to read scientific articles, that's great. So, the signal-to-noise ratio is much better, but you need to understand the landscape in which the people you are advising are, and they are precisely in these cesspools. So, the big picture. Now is a time of colossal opportunities. I urge you, like Andrew, to keep learning, keep digging, and keep building. But there are risks ahead. Remember the movie Titanic? Remember the famous phrase: "Iceberg dead ahead." But immediately before that, there is a scene. I would show it, but I can't due to copyright, where two guys in the crow's nest on the mast are freezing and chatting. Their job is to spot icebergs ahead. Rewatch that moment. They are only discussing how cold they are, and then the team is shown asking, "Excuse me, don't they have binoculars?" And it turns out: "Oh, we left the binoculars at the port." This scene perfectly conveys arrogance. They were so eager to move forward that they didn't want to notice the risks. And even having people whose job it was to watch for dangers, they didn't train them properly and didn't provide them with the necessary equipment. For me, this is a great metaphor for where the industry is now. And there are risks ahead of us. That very word starting with "B," the bubble, that you read about in the news, is a reality. But when I think about bubbles, I remember the dot-com bubble, the early 2000s. Most of you didn't experience it. It was the biggest bubble in history. It burst, but we are still here. And the people who worked with dot-coms correctly, not only survived, they thrived. Amazon, Google, they did everything right. They understood the fundamental principles of how to build an internet business. And when the hype bubble burst, they didn't go down with it. There was a website, I think, pets.com. Their motto was: "Build it and they will come." They ran ads during the Super Bowl, but couldn't handle the traffic. Websites like these vanished when the bubble burst. A similar bubble is likely inevitable. Bubbles always happen. Companies that work correctly will not only survive this moment but will become leaders after it. And the people who work correctly, the people in this room, those who are thinking about how to implement it in their companies and what recommendations to give to businesses, will not only be those who are not laid off during the collapse, but also those who will continue to grow during and after it. If we talk about the structure of any bubble and what I see now in AI, it's a kind of pyramid. At the top is the hype I mentioned. At the bottom are huge venture investments. I'll be blunt, I'm already seeing them start to dry up. It used to be enough to write AI on a slide, and you got venture investments. Then it was enough to do something with large language models, and companies got investments again. Now investors have become much more cautious. I advise many startups and see investment amounts decreasing, priorities changing. This second layer, massive venture money, is already starting to disappear. Unrealistic company valuations, companies without profit, but with sky-high capitalization. We all know these examples. Unrealistic valuations are fueled by hype. Then come "me too" products, when someone does something successful and everyone else jumps on the same bandwagon. We see this everywhere, just like during the dot-com era, and at the very bottom. real value. Perhaps I should have drawn an inverted triangle, because there is little real value yet, but I quickly coded these slides. So here is another example of technical debt that I have taken on. So, there is real value there after all. It's a small but very important core. And it is those who build around this real value who will ultimately survive. Therefore, the direction in which, in my opinion, the AI industry will develop, what skills I would recommend you consider, looks like this. In the next 5 years, we will see a clear stratification or, if you like, a fork. I will call them big AI and small AI. Big AI is what we see today. Giant language models that strive to become even bigger on the path to AGI, Gemini, Claude, OpenAI will continue to scale. The logic of these companies is more is better: either in terms of approaching AGI, or in terms of extracting business value. This is one branch, the other branch is small AI. We all see the growth of so-called open-source models, although I don't like that term. It's more accurate to say models with open weights or models that can be self-hosted. Their number is growing explosively. I recently read that 80% of Y Combinator startups use small models, especially Chinese ones. China is currently leading in the small models segment. Perhaps because they are not as fixated on gigantism as the West. I see this stratification becoming more and more apparent. Perhaps China has a temporary advantage in small models, perhaps not. I'm not going to claim. But the point is that we are moving towards a world where there are models hosted and maintained by someone for you. GPT, Gemini, Claude, and models that you host and use yourself for your specific tasks. Currently, this segment is not yet well-developed, and the bubble may burst here sooner, and here later. And the key skill that I see will be particularly in demand for developers in the next 2-3 years on this side is fine-tuning, i.e., the ability to take a model with open weights and fine-tune it for specific applied tasks. I will give a specific example from my own experience. I work a lot with Hollywood and the film industry. I even managed to sell a film to one studio. It's still in pre-production and will probably stay there forever. But in the process, I realized one important thing. Intellectual property protection in studios is taken to an absurd extreme. Google James Cameron and Avatar lawsuits from people who allegedly sent him a story about blue aliens many years ago and are now demanding billions of dollars because, apparently, he used blue aliens in Avatar. But the level of intellectual property protection in Hollywood is simply incredible. At the same time, the capabilities of most language models are equally incredible. Today, the main focus is on generation, script, stories, visuals, rendering, and so on. But in reality, a much more powerful capability is analysis. Analysis of film synopses, understanding what worked and what didn't, why one film became a hit and another failed, what time of year a film was released and became successful, and when it wasn't. And given how minimal the margins are in the film industry, such analysis has colossal value. But for this, studios must transfer the details of their films to a language model, and they will never do this through GPT, Gemini, or any other external platform, because it means transferring their intellectual property to a third party. And this is where small models come into play. A studio can host a model on-premises, fully control the data, and at the same time, these models are becoming smarter. A 7 billion parameter model today is comparable in intelligence to a 50 billion parameter model a year ago. And in a year, a 7 billion parameter model will be comparable to a 300 billion parameter model from last year. Therefore, we are moving towards building solutions based on small self-hosted models, which are then, yes, trained for specific tasks. This applies to all areas where privacy is critical. This is the stratification I see in AI: an early bubble, in my opinion, in the area of large, non-self-hosted models, a later one in the area of small, locally hosted ones. But in any case, when it comes to your career and how to survive any bubble, focus on the fundamentals, create real solutions, understand the business context, and, most importantly, diversify your skills. Don't become a one-trick pony. I've worked with brilliant engineers who knew one API or one framework perfectly, and then the industry moved on, and they were left behind. And yes, when bubbles burst, the consequences are always roughly the same. Funding disappears, hiring freezes turn into layoffs, projects are shut down, and the market is flooded with talent. So how do you think, how well? So, the question was primarily about Nvidia and generally about hiring for a very narrow, specific scenario. And then a more general question arises: what is more important to you: to become an expert in one narrow field, or to diversify your skills? I will always insist that diversification is still more important, because that one narrow scenario remains effectively one scenario, and you are essentially putting all your eggs in one basket. Nvidia is undoubtedly a fantastic company. There are no complaints there at all. Working there is a dream. But if you bet everything on Nvidia and ultimately don't get in, what then? Therefore, I believe that if you are truly interested in something, going deep is great. But knowing only that one thing is risky. I always advocate for diversification. And when I say diversification, I don't just mean choosing between large language models, computer vision, or something similar. That's only part of the picture. But for me, knowing models and being able to use them is a universal skill. Skill diversification goes beyond that. In addition, you need to be able to think: "Okay, what about creating applications based on this? What does scaling an application look like? What does software development look like in this case? And what about user experience and user interaction skills? After all, creating a beautiful application is good, but if no one can use it, then yes. I'm looking at you, Microsoft Office. That's what I mean by diversification. Even in your Nvidia example, it's important to be able to go beyond a specific case and demonstrate skills in other valuable areas. And since we are limited in time, I will briefly say, I am a staunch advocate of small AI. I believe that small AI is the next big breakthrough because we are moving towards a world where AI will be literally everywhere and simultaneously. And this, by the way, is part of my work at ARM. There is a traditional view, and it's interesting that you mentioned Nvidia, because there is a traditional view that computing platforms are CPU + GPU when it comes to AI. CPU is universal, GPU is specialized, but this view is starting to change. But, for example, in the mobile sphere, huge innovations are happening with a technology called Scalable Matrix Extensions. The essence of SM is to move AI workloads directly to the central processing unit (CPU). The leaders here are Chinese phone manufacturers Vivo and Oppo, who recently released devices with chips supporting SM. And this is a magical thing. Firstly, there is no need for a separate external chip that consumes additional energy and takes up space just to run AI tasks. Secondly, the CPU is by its nature an energy-efficient component, and the ability to perform AI workloads directly on it opens up entirely new use cases. I will give a specific example. There is a company called Alipay. They had an application, and we've all seen similar ones, where you can search by your photos. Show me places where I ate sushi, for example, and then automatically create a slideshow. Usually, this requires a backend. Your photos are stored in Google Photos, Apple Photos, or somewhere else, and the model runs on the server. Alipay looked at this and said, "There are three problems here. First: privacy. You are transferring your photos to a third party. Second: latency. You need to upload the photo, send a request, wait for processing on the server, and then download the result. And third: cost. Deploying and supporting a cloud service takes time and money. Therefore, they decided to move all this directly to the device. The idea is for the model to run locally and search for the desired images directly in the phone's memory. No delays, no data transfer outside. And from a business perspective, savings on infrastructure. As a result, AI runs directly on the CPU. Apple is also actively investing in SM. This is clearly visible if you've ever watched WWDC or similar presentations when they talk about new M-series chips, neural cores, and similar technologies, it's all part of the same idea. It's about moving away from the traditional view that a GPU is absolutely necessary for AI to work. And this is precisely the trend that the whole world is moving towards now. Apple is perhaps one of the leaders in this direction. I am very optimistic about Apple and Apple Intelligence for precisely this reason. If you look at it from the perspective of AI and follow this trend to its logical conclusion, it becomes clear that as models shrink and embedded intelligence appears literally everywhere, this is no longer fantasy or science fiction, but a reality that we will see very soon. The idea of AI convergence, thanks to smaller models becoming smarter and energy-efficient devices capable of running them, is already being realized before our eyes. I see huge opportunities in this. And the last point. I want to return to agents for a moment. I often say that artificial intelligence has a hidden side, what I call artificial understanding. When you start using models not just for generation, but for them to understand something for you, and then create new things based on that understanding, you gain real superpowers. You become much more effective than before, whether it's writing code or creating any other content. I will show one short example to summarize. Earlier, I talked about video generation. In this photo. Oops, sorry, weak internet connection lost. In this photo, my son, he plays hockey. I took this shot and thought, "Okay, and I'm excellent at writing prompts." I came up with a good prompt, as I thought. He's in the moment of shooting the puck, with beautiful ice sparks on the stick. And I asked the model to generate the moment where he scores a goal. What do you think happened? Want to see? Let's see. It wasn't the video, but the essence is the same. Due to a bad prompt or a poor understanding of my intent, to use agent terminology, this is what happened. The rink where he was training is actually empty. There are no spectators. If you rewind a bit and look at the top right corner, there's a zone where they store garbage. But I didn't know that either, and decided it was a full-fledged arena, and added spectators. As a result, he shoots the puck completely wide of the goal, but everyone is cheering happily. He has two sticks instead of one for some reason. And even his name was mixed up. All because I didn't use the agent approach. I didn't go through the steps. A. Understand my intent. B. Determine what tools are available. In this case, VO C. Understand the nuances of their use, create a plan, build the correct prompt, execute, and then analyze the result. I am currently advising a startup that is involved in creating films using AI. I want to show a small fragment of the project we are working on. The key idea here is that if you want to get convincing performances from virtual actors, you need emotions. These emotions need to be not just generated, but also integrated into the context of the entire story. Because when you create a video from a prompt, you get an eight-second fragment. And this fragment needs to know what is happening in the rest of the plot. So, if I click here, this demo is a bit wooden for now. My actor friends laugh at such performances. But feel the difference between that non-agent prompt with the hockey player and this result. I think I'll be able to go to the quiz in Pap. >> They just shut me down. I was so close, but they wouldn't even listen. They never listen. The idea is that by breaking down the process into agent steps, which I talked about earlier, I was able to use the same engine that previously gave a failed result and achieve a working solution, including conveying emotions, which I just mentioned. We've gone a bit over time,
So, I apologize. I am ready to answer questions, if there are any. I see that Andrew is here too, he is in the back. And I just want to say a huge thank you for your attention. I truly appreciate it very much. Yes. >> To what extent is this progress due to the workflow? And to what extent is it a problem of training data bias, where for every two or three hockey fields without people, there are thousands of filled stands? Yes. Excellent question, I'll just repeat it for the record. What part of the improved result is specifically related to the use of an agent-based workflow? And what part is due to the fact that the training data for the unsuccessful example simply lacked hockey data? I am not comparing fully comparable things here, so I am speaking more on a feeling level. But when I looked at it more closely and broke down the process into steps, I told myself: "Okay, I am creating scenes this way." It became obvious that the scenes I was creating directly without any structure, without an agent-based approach, and without what I call artificial understanding, turned out simply terrible. When I broke the process down into steps, everything changed. In one scene, a girl is sitting on a bench, she is upset, a guy approaches her and wants to comfort her. I fed this to a large language model along with the entire story and my constraints. The shot must be exactly 8 seconds long. The dialogue must be clear, and so on. And when the large language model understood my intention, it composed a prompt that was much more verbose and detailed than I could have ever written. The language model had an understanding of what makes a shot good, what angle is better, and how to convey emotions. This understanding was much deeper than mine. I could have spent hours trying to describe it. So the first step in the agent cycle, where the model understands my intention for me, became key. The second step is tools. I explicitly specified which video tool I would use. As a large language model, I used Geminii and hoped that Gini was familiar with VO. This helped to take into account all the nuances of working specifically with Vio. I learned, for example, that Vio struggles with dynamic scenes but is very good at slow camera movements to convey emotions, as you saw in the example. The language model knew this because I declared Viо as my tool. Then it composed and refined the prompt. The third stage is generation itself. Creating a video in Vio costs approximately $2 to $3 for four variations in credits. The last thing I wanted was to generate hundreds of takes and waste money, but all the token costs for understanding the intention and creating the plan paid off. The agent produced excellent results, if not on the first try, then on the second or third. So even without a direct comparison, I am confident that such a workflow works wonderfully. Are there any more questions? Questions, comments? Yes, at the back of the room. What has surprised me most in the AI industry over the years? Good question. Perhaps what surprised me most, and perhaps shouldn't have, is how much hype has taken over. I sincerely believed that people in high positions, making decisions, would be better able to distinguish real signal from noise. I was also struck by the pursuit of immediate profit at the expense of long-term prospects. I'll tell you one story. After Andrew and I released the TensorFlow specializations on Coursera, Google launched a professional certificate. The idea was to conduct a rigorous exam. If you passed it, it became a prestigious confirmation of skills and really helped in finding a job. At that time, TensorFlow was at the peak of demand. Supporting this program cost Google $100,000 a year, which for such a company is a drop in the ocean. But the reputational effect was colossal. I remember the story of a guy from Syria. He told it in Google's promotional materials. You know how difficult the war was there. He became one of the first in Syria to receive this certificate. And it pulled him out of poverty. He was able to move to Germany and get a job at a large German firm. I met him at an event in Amsterdam, and he told me that now, thanks to this job, he can support his family and move them from the war zone to a safe place. And all this thanks to him figuring it out. You see, there were thousands of such inspiring stories. But what surprised me is that the program was closed. Why? Because it didn't bring the company direct income. We deliberately made it self-sustaining so that the exam fee was low. As a result, it didn't break even, costing the company $100-150,000 a year. And the project was shut down. It's terribly disappointing, considering how much benefit it brought. But this is what surprises me most. I am also inspired by people who achieve success in AI, even though you would never expect it from them. Let me tell you another story. I have a close friend. I showed photos with hockey. So, he is a former professional hockey player. Are there any hockey fans here? You know, it's a tough sport, there are a lot of fights and skirmishes. He dropped out of school at 13 to focus on his career. He always says he's the dumbest person in the world because he has no education. We are complete opposites, which is why we are friends. He left the sport due to a concussion and now runs a non-profit organization. They build ice rinks. About three years ago, we were drinking beer, and he asked: "Listen, tell me about AI and this ChatGPT. Is it really worth it?" I started explaining: "Yes, it's cool." And so on. It was a trick question. I didn't understand why at the time. It turned out that in his non-profit, he needs to present to the board of directors every quarter. Reports on work to receive funding. Even non-profit organizations need money. He was spending over $150,000 a year on consultants who collected data from various sources. They have data everywhere: sensors in the compressor that cools the ice, a bunch of spreadsheets, dashboards. He's not a tech guy at all, but he needed to process all this, and he decided to conduct an experiment, he tried to create a report using ChatGPT. That's why he was asking if it was a good thing. We discussed it, and I looked at the results. He uploaded spreadsheets, PDF files, and asked to compile a report. Now it takes him about 2 hours with ChatGPT, and the result is brilliant. The $150,000 a year he saves on consultants now goes to children from low-income families for skates, equipment, and hockey lessons. The money went from expensive consulting firms to people. And this guy, who calls himself the dumbest in the world, I hope he's not watching this video, did it himself. I told him: "Congratulations, you're a developer now." He didn't like that. >> [laughter] >> But it's precisely these things that make me happy. The superpower ended up in the hands of a person who is not at all involved in technology but was able to build a solution that saves $150,000 a year. Such things are always pleasantly surprising. Yes, I will answer you in the next one. Yes, >> for us engineers, it's easier to navigate this. We understand the essence, but what about people who don't have such a background? I'll repeat the question for the video. It's easy for us to see the signal through the hype noise. But what about everyone else? I believe that this is where our opportunity lies to become trusted advisors for them. The main problem with hype right now is that all mechanisms online reward engagement, not substance. The first step is to learn to see through it. Like my hockey player friend, he saw all this noise but didn't want to risk his career. He needed advice from the outside, what he was doing right and what he was doing wrong. It is precisely the ability to position yourself as a trusted advisor, without repeating the mistakes that unprepared people easily fall into, that is the key. And it's also important to understand that the average person is actually very smart, even if they are not an expert in a particular field. We need to leverage this intelligence, help it develop, guide people through difficult moments, and give them the opportunity to shine in what they are truly good at. I saw a question there. Okay. So, AI and machine learning for scientific research. Where is it a good idea, and where should one be cautious? My intuition suggests that it's always a good idea. There's nothing wrong with using available tools, but you always need to double-check results and align your expectations with reality. I have always been an advocate for automating research as much as possible. Many years ago, I studied physics and was quite successful in the lab precisely because I automated things on the computer that others did manually with pen and paper. This allowed me to move faster. So I'm biased on this issue. But I would say, for most research, use the most powerful tools available, but manage your expectations. There's a story on this topic in a quiz format. The poorest country in Western Europe. Does anyone know what Western Europe is? It's Wales. I was studying in Wales at the time. Later, I returned there to give a lecture at the university and met a researcher who was studying brain cancer using CT scans and various imaging techniques. I asked him: "What is your biggest problem? What is hindering your research the most?" This was about 8 years ago. His answer was the lack of access to GPUs. To train and run models, he needed graphics processors, and in his department, there was one GPU for 10 researchers. This meant that each person got access to it for half a day from Monday to Friday. His time was Tuesday afternoon. All other times he spent preparing data and models for running. And then on Tuesday afternoon, when his turn came, he would run the training, and all he could do was hope to get the desired results in those few hours, otherwise he would have to wait another week. Then I showed him Google Colab. Has anyone used Colab? You can get GPUs in the cloud for free there. The poor guy had a paradigm shift. I took out my phone and showed him a notebook in Google Colab that was training a model directly from my smartphone. For him, it changed everything. It turned out that even the free Colab plan gave him more power than that one GPU for ten people. Machine learning was a critical part of his work, but access to it was limited. Removing this barrier gave a powerful boost to his research. I don't know how it all ended or what he achieved. Several years have passed since then, but this story immediately came to mind after your question. Are there any more questions? Ask anything. Yes, over here, can AI be a force for social equality or social inequality? I think the answer is yes. [laughter] It can be both, and neither, and neither at the same time. Ultimately, any tool can be used for any purpose. Therefore, it is important to educate and inspire people to use technology for good. Government regulation can only do part of the job, and sometimes it creates more problems than it solves. In my life, I am guided by the principle: assume good intentions, but prepare for the bad. AI and everything else are the same. Everything I do and advise, I do based on the presumption of good intentions, hoping that people will use it for good deeds. But at the same time, you need to be prepared for abuse. The bad examples I showed earlier are more the result of mistakes made with good intentions, rather than malicious intent. So my only advice in this regard is: always assume good intentions, but be prepared for bad ones. AI itself has no choice, right? It all depends on how people use it. Andrew, do you want to say a closing word? Okay. Thank you, Andrew. Thank you, everyone.