Transcription
So this is how a recruiter interview these days in AI is. They are testing you for pretty much what you have done. Have you worked on the latest greatest stuff? Then they are looking for challenge in shipping. Do you have customer insights in the area? And then they are looking for how are you communicating your simple ideas across the teams.
Okay, then let's get started. Thank you for joining. If you're joining for the first time, I think the first few slides make a lot of sense to you. So, let me go through them really quick. My name is Mahesh. I was a product leader at Google where I build back office agents that automated support, legal, and provide a platform for our finance team to build agents. Before that, I worked at AWS where I built bedrock and figured out how can we scale large models on inside AWS and grow them. And before that I worked at Facebook where I was working in the distributed recommendation team, Infra team mostly and figuring out how to scale our training stack from billion to trillion scale. also worked in PyTorch where I shipped PyTorch 2.0 moving the distributed training from upper layers to compiler so that OpenAI can train large models like GPT 3.5. Uh and before that I worked a lot of time in Microsoft. I was an engineer for 15 years. Last six years I'm doing what most of you are doing which is product management and I'm teaching for a long time. Uh that's the another fun fact that uh I t taught three years just in schools uh in Microsoft. It was a teles program where I would just go and teach high school students and then I was teaching all along because my visa never allowed me to teach outside. So I was teaching inside these companies. So I had a course if you're in Facebook you can go and search for recommendation system course for PMS and I built that. But 2023 I got to teach broadly and uh I think we are teaching our 12th cohort combining my previous stint as well. So a lot of experience in teaching a lot of experience in AI and lot of experience of having fun with community like all of you guys. Uh if you are open I am still open to connect with you. So send me your request and I am still accepting them. So this is my LinkedIn and I would love to build this community. So I share it. A lot of people reach out to me and I reject most of the direct connection request but if you send me request from this session happy to connect.
Let's get started. What's the interviews test beyond AI knowledge? So they are asking you for your AI product sense. Have you built stuff in AI? What you have done in AI? But they are asking you more stuff like technical stuff and we will cover most of that today. Uh what has changed since 2024 in 2026? what's new in the technical side. Hopefully I can touch on two concepts today that are coming again and again in interviews or at least people are curious about them. That's the latest greatest thing happening in our industry in 2026. I won't touch on that. Uh I will also explain why most of the PMS are failing system design and how to avoid that. I will take two case studies on that. That's where I will focus today session mostly doing AI system design and how to crack those interviews and I will give you rubric or I will give you how to go step by step or give you some tips on how to conduct yourself during interviews especially technical ones. Okay. So if you're here for non-technical stuff today maybe do something else. Today is going to be a little technical but uh hopefully we make it fun. Uh so what are they testing beyond AI knowledge right? Uh so AIPM interviews if you're going for AIPM interviews they are divided into two things. uh it's technical or it's product sense. In technical if you look at we are looking for your concept and understanding. Do you understand basic things like how a large language model is trained? How it works? What is the limitations? And then we are looking for system design. And then we are looking for ecosystem understanding. And then we are looking for have you ever shipped AI specific stuff which is have you done a discovery of AI project? Have you actually created road maps or planned AI things and then have you gone ahead and did evaluations pricing or GTM. So most of the things generally what happens is you will see in this area but recently I have seen lot of questions in 2026 around system design in technical. So I thought maybe I do a session and walk you through. Uh by the way uh uh because of all the things I'm doing on LinkedIn and this course uh I'm getting no calls from recruiters these days. Earlier every two weeks I was supposed to get a call and I used to talk to them because I just have so much fun talking to recruiters and giving interviews. My wife told me that at some point that became my passion. That was the entertainment for me. Uh so I did more than 300 or so interviews uh when I was like at uh at Google and others just to know what's happening in industry. But uh I spoke to a recruiter recently and let me share what they are asking. These are the recruiters by the way. The first recruiter call that's happening and I think some of you can resonate. The recruiter call these days is not like how hey what is your expectation what you did last uh what was your job and here is the job I will send you to hiring manager manager. What's happening is the recruiter call is the first interview call and I will walk you through what this recruiter talked to me. So the recruiter went ahead this was my story. So here is what happened. Can you share what AI projects you have done in past two years? So I shared okay I built this thing at Google. I build agents and a platform for agents. We automated support and we automated our legal function. Oh. Then he said, okay, you build the support for Google. Was it a chatbot or agents? So I was like, okay, I didn't know like where that going. So I said, okay, these were agents, not just chatbots. And then he was like okay then walk me through what are the components you used which model you used and have you used rag or not and any other components that you used in your app. So then I explained to him that hey we use Gemini or we did this rag or we did a hybrid with Google search and our own documentation and I explained all the components. Then he was like okay what are the top challenges you faced in shipping these to customers and now it's another broad question and then I said okay we had GCP support we have GWS support in GWS support we faced the issue that we didn't had lot of interaction the customer wanted the answer right away and if we did more interactions or if we went to do multiple chat rounds the customer lost interest while in GCP it's a very technical customer it's a developer persona so they had complex problems and when we gave simple answers or gave them documentation they ran away so how the to engage the customer so that we can solve their problems at different level was the biggest challenge and then I told him a couple of ways how we solved it and then he was like how did you communicate AI specific details to other teams like growth or UX teams and here he's trying to figure out do I simplify things do I make it complex and other things and then after that he said by the way the next round will be with the VP he's very technical so prepare to answer questions on how you improve the prompts rag and secured agents best of luck so this is how uh a recruiter interview these days in AI is if you look at it they are testing you for pretty much what you have done have you worked on the latest greatest stuff At that time it is agents. Then they are looking for challenge in shipping. Do you have customer insights in the area? And then they are looking for how are you communicating your simple ideas across the teams. So these are three or four buckets that you will see come to you in every interview that you do as well.
So what are they checking you for? Right. So what's new right earlier and this is what I'm seeing again and I'm very lucky by the way that most of you reach out to me when you have interviews especially the people who have taken my cohort and have a relationship with me uh although all more people reach out to me but I only respond to people who have taken the cohort uh but the idea is that also they are not testing in AI specifically for typical product sense questions like hey improve a product or go ahead and set the northstar metrics or what the dashboard should look like for this product. Those questions are not getting asked in AI. In AI, they are asking for what you have done in AI and what are the challenges you have faced in your AI projects and how you went ahead and solved those problems. So that's a new trend that's happening especially for AI specific roles. By the way, they ask you technical questions, but they're not expecting you to code. Although people are talking about vibe coding and all, I have seen very less of that. They might give you an assignment to do in the beginning, but during the interview, they don't want you to open like you know Visual Studio Code and start coding. That's not what they are looking for. But they are looking you to explain the design. Did you use rag or not? If you use rag, then what kind of rag? what was the problem with the rag, how they got fixed. So a broader highlevel design I think is becoming a table stake requirement for these interviews and they're not asking you all your star or like personal life challenges. They are looking for how you build customers for the first product, how you retain the customer if you build a popular tool and then they are asking how you actually build ROI for the business. This is becoming the number one questions in 2026. Again, we invited one of my friend who works for Oracle cloud and he was telling we invited him in our boot camp where we train people for interviews and uh he was saying that these days PMs are coming with lot of technical diarrhea. So when he talks to them the PM just start talking a lot of technical stuff while he wants to uh hear is have they brought ROI have they actually built the business and that's I think most of us forget in AI because we learned this new cool stuff multi- aents long horizon jobs uh agentic rag we forget that our job is still to build customers for business and once you have the customer build profitable companies or build the business. So make sure you don't forget this last piece because that's what you are checked on.
So product sense design. Having said design, let's talk about system design because that's new and in AI they are asking you not like older system design questions. They are asking you or checking you for the latest and greatest system design question like AI system design and not traditional system design. So let's see how can you handle specific system design that's the focus of today's call. So if you just joined you miss nothing. So how are people checking you for or what is their rubric? They want to make sure that you understand how these models work. They want to understand that the model is just one piece of the solution. What are the other things like tool calling rag or knowledge handling? How you handle memory? how you build good guard rails and how you build multi- aent systems which can interact. So this is like another way to check on your system design and then they will ask you a lot of questions on how to build this. So if they ask you a simple question like hey can and most of you have created CVS with AI now and they will pick something on your CV for an example they will ask you how you design a multi- aent system or can you walk me through your multi-agent system diagram or design a system with multi- aents what are the main components of multi-agent system okay orchestration we had focus Yes, we needed coordination. Anything else? We had >> guardrails. >> Tools, guardrails, >> guardrails, circuit breakers. >> Guard rails. >> Memory sharing. >> Can you memory sharing? >> Yep. >> I think let's put it here. >> Observability. >> What is that? Sorry again. >> Observability. >> Observability. Great >> feedback >> or reliability >> explanability uh explanability >> and I guess >> again again again again again again I need to bring you back uh I put the component so all of you can feedback >> uh great so I know it's easier when I ask you for components but the question is can you just walk me through a highle design on a multi- aent system how it worked okay let's do an agent the user ask a question where should it So which component first? >> Orchestration. Okay. >> Start with an >> okay. Let's say it goes to the orchestration. The orchestration decides what? >> What will it have access to? >> Basically agents roles and responsibilities. That's the first task. >> Specialist worker. >> Yeah. So now you have agents. Agent one, agent two. So if it is a major multi- aent system, you will have lot of agents. Okay. agents three and then can I put a human as a agent two so human in loop can I call that also which can be our team or the customer okay I have orchestration based on customer question let's take an example we can take the legal example the customer uploads a contract and wants to know what are the risks in this contract if they sign it okay the orchestration gets it the orchestration decides cost agent is that okay I need to take this contract it's a PDF document the customer uploaded I need to convert it into text oh by the way I realized during that this agent said oh I found what should I get also here another thing another component so I called the first agent >> great I can put a memory layer for coordination this is this And here it updates that hey I found in this PDF some documents some pictures some tables which I can't process. So the orchestration goes and it updates the memory. Every time the memory get updated orchestration reads it and make sure that hey checks if the job is done if we are done or not. If not then it goes back and says okay I have new information I need to call a separate agent which can read tables and images and dump the data. Okay now I have data from the text plus the tables plus the images. Then I then once it is there then it goes and asks for the next agent which is the agent which goes and find the key terms or find the key clauses in your contract. Then the next agent which find risks then the next agent which checks and it goes on and on. So the simple orchestration highle diagram is you can have a memory as a coordination you can have different agents with different focus and you can have an orchestration layer which goes ahead and finds and does specific or different things for you. What is the great thing here? like what a what is a good or bad response look like for a simple question like this five six if you just talked about what we talked about in the beginning if you just threw all the things that hey highle diagram it will have orchestration it will have reflection it will have this and it will have that I will give you five and six but if you are able to go and walk me through the key concepts and build up to get build them together as a system of record which is how orchestration works, how memory works, how different agents are getting called. I will give you seven and eight. But if you take an example and if you talk about a user, what the user does when the user comes back and weave it together the technical as well as the nontechnical things in your system, you are getting 10 out of 10. So that's what is expected from you. When people ask you a technical question or a system design question, just don't go after saying, "Hey, here is my all my technical skills." Try to marry it with an example and then as a lot of people raise their hand once they ask you questions, you can go back and forth with an example and double click on anything. But start with a very basic thing. Don't go in too much details of reflection. Don't go in too much detail of how the rack component works in this. Which tools are getting connected just show multi- aents lot of agents orchestration is needed to orchestrate memory is needed to coordinate and stop there and then when they ask you questions just build a very simple use case and then build on top of it is your goal. Good. I I I wish good.
Okay. Then what are the three types of questions people are asking in this? I have seen mean I think last three weeks I have heard maybe seven calls around different interviews. So in system design you are getting three archetypes. Either they will ask you to explain something that's on your CV. That's the most easy for me to ask you or easy for these interviewers and should be easy for you to answer as well. Or they are going to ask you a system design of a new problem which is just I created maybe I will say hey let's design a multi-agent system for contract processing and they will ask you questions on that. or they will show you a simple design diagram of a multi- aent system and they will ask you or say one problem into it and they will ask you to go and improve it. Okay. So make sure that in your stories or in your head you have three examples which can do these things or you have practiced through these three things in your CVS. If you are writing anything that is AI or anything that is complex which uses cloud make sure you have a system diagram handy so that you can walk through it you can design a new system that is a muscle and hopefully I will give you some tips how to have this or how to make improvement this comes by practice so make sure you have done some practice mocks around all these two and you have all of this already written the first one is the easiest part if somebody asks to system design of your latest project. Hopefully you understand it. Let me give you some tips on that. One is make sure you have the highle diagram already written. So this is one thing I build for Microsoft and uh this is a vision AI factory AI stuff where I can ingest lot of camera streams and then process the frames generate insights and send triggers over teams or slack. I did it in 2021 which sounds like at this time dinosaur age but uh if people ask me questions this was people asked me this question when I was interviewing at Facebook they said hey can you walk me through this uh diagram or how this system worked and by the way when you go to that page I already put this page there on my blog and here what how it works is there are a lot of cameras we get the camera stream. We have a module which analyzes these, cuts the video frame by frame, sends it to the orchestration for inference which is nothing. Inference is nothing but processing the video. We send it to the model and the model gives us what is there in the picture. We get the results. We send them results back to IoT hub. From there we have an app and we decide whether we want to store it in our CRM. We put it for time series or it's an alert we need to give to the user over teams. So now you see that I can explain all this in like 2 minutes because I have it all figured out before even getting on a call. And now they can ask me questions. The kind of questions I have got here is hey why are you using two modules? Why can't the orchestration also run the inference? So I can answer that saying hey when we did it we wanted to expose people to bring their own models but our orchestration had a lot of customer IP logic and it was very hard for other developers to build or understand our code that's why we separated it outside and you can ask me any questions on this and I will be able to answer it because this is my system diagram or this is my system design and I have worked on it. So make sure you understand your own products that you have worked on and you can explain to the interviewer when they ask you or cross question you and you have done the work up front by writing a blog or have deduced this or written down how this system works so that you can just read through it and you can ask me any questions on this I should be able to answer.
Let's go to the second one right which is people ask you to do a system design of a new problem. They might give you a legal use case which we were just talking. So if we have that context let's continue. So they might say hey how will you use knowledge graphs in a world like this? Okay people ask you that hey have you used knowledge graphs? How you use knowledge graphs? Then all you can do is show or build this live with them which is start with very simple things first before you go into details you can just start with basic diagrams saying hey the user ask a question gives a contract it goes to LLM and we get the answer very simple that's your first step then you bring a problem then you say oh this one or they will say oh what if the size of contract is more than the context the model can take then you say okay great then what we are done is we did what we did is we did rag after this so this is a rag diagram if you don't know rag we have done lot of sessions on rag it's on our YouTube but the idea is very simple instead of sending the documents you put all their documents here or the document they upload first goes gets into smaller smaller pages that converts into something called a database and then when the user sends you a query you only get the pages which are relevant to the question and then you only put relevant information and then you can solve the problem. Then you say oh this problem worked for simple queries but for complex queries like interest rate it didn't worked. Okay why didn't worked? Because interest rate is distributed across page number 30, 10, 22 and 70. Because to calculate the interest rate, you have to first calculate how the interest rate is supposed to be calculated. It's a multihop problem. First you have to calculate the benchmark. Then what is the benchmark? Then the index floor of that benchmark. You also need to get the spread. And then you have to sum it up to come to interest rate. If you just ask what is the interest rate, it will be wrong 90% to 95% of the time with a simple rag system. To solve this problem, I used so this is the problem when you get the chunks you miss the chunks that you need to solve the problem. So what we did is we first created a graph where we if the user ask a question we figure out what are the terms that are relevant for interest rate. So interest rate is made up of benchmark spread and all that is stored in our knowledge graph. First we get those out and then we go and retrieve the relevant chunks and this is how we use knowledge graph. Then I can go and talk about building knowledge graphs. So what is important for you is you have to show me iterations. Don't go and try to create a very complex system diagram for me. Try to create a very simple one first. Just the LLM you trying to solve a problem. Then you go and say hey what are the challenges with this system diagram or they will give you the challenge. Then you build the next layer. Then you build the next layer. And make sure you are creating the tension between you and the interviewer because if you can just go and give everything first it will take 20 25 minutes to write a complex system diagram and by the time your interview will be over and if they're not interested in that you might have wasted their time and your time. So go start simple and then rationalize why you choose what you chose step by step like I was explaining to you and then give me why cost latency what are the challenges in knowledge graph why not knowledge graph for everything just for these complex one you should have that drawbacks of each and every choice so that's pretty much it right and make sure you are pausing and ensure that your audience or your interviewer is following along. Make sure you can explain each step and make sure you create tension in the system and then release it which is like hey I got these three answers correctly but I didn't get the interest rate correctly. Why? Because interest rate is made up of these two things because this three things are distributed across these 20 pages. So now you know now people are interested in solving or listening to you further.
So final thing what are we judging you on when you are interviewing? We are judging how much AI concept you understand what are the secrets or what are the things that you have learned by doing not by just reading somebody's else blogs and you're not getting judged on how good you are at coding how many things you have built how many things you have yourself taken to production nobody's asking you to have a research level understanding of AI concepts either like what is vanishing gradient Although you know it, nobody's checking you. I will give you bonus points, but that's not table stakes. And nobody's asking or relying on you to scale a system or take a system and scale it to thousands or millions of users because I have developers for that. So make sure you focus less here and you focus more here in your interviews. And with that said, one last thing I want to land is that as I talked about few system diagrams, you should have all of these system design diagrams in your head or you should have practiced. With that said, there are a lot of labs we do in the cohort or in the course. We have enhanced it with now long horizon labs and the L labs with observability. So you can just open if something like this comes up you can just open one of the labs and explain people like for multi- aent systems we already have a lab where we have a multi- aent system and you can just explain it by opening a diagram like this you can even show a live demo and that's the kind of things you need to do before because at the day of interview if you call me I can't help you I will just point you to these labs so that's why we have done this cohort where we spend six weeks with you and help you build all these kind of system design in your head, show you the tradeoffs and teach you all the tools that you need to be successful. By the way, we are the number one course on Maven. This is the team took the screenshot and they want to talk about it. So, thank you for all your help and all your support. uh you have made me uh to get this far and I don't know what I can do to keep staying this far but uh thanks a lot we are the number one course we are starting our next cohort and if you join today we are happy to give you $ dollar 500 off which I think we are not giving anymore but just for this cohort just the people here if you do it for next 8 hours I think we are planning to stop enrollments as well I think we're done I don't know how many seats we are left with but most of it is gone. So if you are still on fence please join the cohort. We will talk about how to build AI projects, give you AI product sense but also build your muscle on system design, technical labs and how to actually build and ship AI based projects. After that we will spend three weeks in a boot camp where we'll prepare you for interviews what to expect. This was a premiere on system design but we will discuss a lot. We'll also get lot of my friends from industry to talk to you and see what they are looking for when they interview. And next time I'm going to talk technical stuff. So we're going to build real agentic systems uh concept to co-pilots and we will build agents with long horizon. And hopefully I can break cloud code for you on two things. how long horizon jobs are run in cloud code and also show you how the multi the agent or sub aent system works inside cloud code. After that we have a session with Nancy. Uh Nancy is my friend from Facebook. She teaches a people how to crack product sense and execution interviews. So she's coming and she will talk about product sense and execution. not AI but basics product science and execution because that still is barrier for a lot of people. So that's happening on 25th 20th Feb. If you want to subscribe or get all the sessions that I do or my team does please fill this form and we will invite you to all the future sessions. We will also send you a weekly newsletter so you can stay on top of what is relevant rather than lot of lot of noise. This is one signal newsletter that only thing you need to read to stay up to date with AI for PMS. These are all the resources. Having said that, thanks a lot.