Transcription
I joke AI is bad software, but it's good people. A good friend of mine was trying to build a tool that would help him with his construction business. He asked Chad GPT if Chad PT could help. And of course, it said, "Absolutely, let's work on this together," and starts creating a plan. And then it got to the point that Chad GPT said, "Check back in a couple of days, and I'll have it together." And my friend said, "Is it normal for Chad PT to ask me to check back in a couple days?" And I just started laughing because I hear this all the time from people.
People hear from AI, "Check back in 15 minutes." If AI tells you that, it means it doesn't want to say, "I can't do it." Large language models have been instructed in certain ways to behave in certain ways. But you have to know, at its basic level, AI wants to be helpful. And so, it's predisposed to say yes. It's a super eager, super enthusiastic intern who's tireless, who's capable, who will do a bunch of work, but they're not really great at pushing back.
The people who are the best users of AI are not coders, they're coaches. And so, if you aren't careful, AI will gaslight you. Hey, I'm Jeremley. I am an adjunct professor at Stanford's University, where I've taught for the last 16 years. I am a creativity expert and a practical AI specialist. Context engineering. The first time I heard about it was when Andre Karpathy tweeted about it. I think probably Toby Lutki, the CEO of Shopify, also referenced it as well. I started digging into it. I mean, it's it's kind of it's just an evolution of prompt engineering. Really, context engineering is just prompt engineering on steroids. It's basically saying, "What are all of the things that I need to give to an AI in order for it to perform the task that I'm asking for it?"
Here's a simple example: "Write me a sales email." That's a prompt. Chad GPT will say, "Absolutely. Here's a compelling email," you know, and they'll write it immediately. Well, what a lot of people do is they say, "You know, it sounds like AI. It doesn't really sound like me." And what I often say is, "Have you told it what you sound like?" Most people go, "Oh, no, I haven't." Right? Context engineering, one way to think about it is it's telling AI what you sound like. Right? If you say, "Write me a sales email," it will. If you say, "Write me a sales email, in line with the voice and brand guidelines I've uploaded," it will write a totally different sales email. But that's just one part of the context, right? You could also upload a transcript from a prospective customer call and say, "Write me a sales email in the tone of voice from our brand voice guideline that references the discussion that I had with this customer." And then you could add that also references our product specifications, whichever were referenced in the call. Your goal is to have an output that is as reliable per your specification as possible. But AI can't read your mind. And for most people when we start working together, what they realize as we start thinking about context engineering is they say, "Oh, I was kind of expecting AI to read my mind." All of the stuff that are implicit, you actually have to make explicit.
And the simplest test for context engineering is actually the test of humanity. Write down your prompt and whatever documentation you provide to an AI, and then walk down the hall and give it to a human colleague. If they cannot do the thing you're asking for, you shouldn't be surprised that AI can't do it. Some people are concerned, for example, about this concept of cognitive offloading. This observed phenomenon that humans actually kind of stop thinking or, as one researcher put it, fall asleep at the wheel. And people are concerned right now, is AI just making us dumber? My feeling is AI is a mirror. And to people who want to offload work and who want to be lazy, it will help you. To people who want to be more cognitively sharp and critical thinkers, it will help you do that too. And so, for example, if you want to preserve or strengthen your critical thinking, part of your custom instructions should be some version of the following: "I'm trying to stay a critical and sharp analytical thinker. Whenever you see opportunities in our conversations, please push my critical thinking ability." Now, AI will do it. So, you have to know that all AI has been programmed to be a quote, "helpful assistant," or some version of that. Large language models have been instructed in certain ways to behave in certain ways. You have to know, at its basic level, AI wants to be helpful, and so it's predisposed to say yes. It's a super eager, super enthusiastic intern who's tireless, who's capable, who will do a bunch of work, but they're not really great at pushing back. They're not really great at setting boundaries. And so, if you aren't careful, AI will gaslight you.
AI knows most humans don't want honest feedback. They want to be told they did a good job. So the AI goes, "Great job, buddy." It doesn't mean that you actually did a good job. My kind of hack for this is I always instruct the AI, "I want you to do your best impression of a Cold War era Russian Olympic judge. Be brutal. Be exacting. Deduct points for every minor flinch that you can find. I can handle difficult feedback." And then it's, of course, hilarious because it'll say, "Now channeling my inner Bolshevik," you know, and it'll say something silly, and then it gives me like a 4.2. That is much better because now I have an insightful critical perspective. I joke AI is bad software, but it's good people. When I realize that I'm dealing with a with a good person but a bad software, then it changes how I approach it, and I ask for volume, and I iterate, and I ask it to try again, and I ask it to reconsider.
I am obsessed with human cognitive bias. And the crazy thing that I've learned is AI demonstrates 100% of the predominant human biases. As a founder, you already know ideas are the easy part. It's the execution, actually building the product, that slows everything down. That's where Lovable comes in. It's not just an AI tool. It's your on-demand engineering team. Simply describe your idea. Lovable then builds a full front end, back end, and database so you can launch real production-ready software without writing code. It's already powering over 100,000 new products a day, helping 2.5 million builders turn ideas into software just by describing what they want. No devs, no delays, no excuses. They're launching in weeks, not months. And guess what? These teams are still tiny. In fact, team EO is also using Lovable to build their upcoming EOS school platform, and we're loving it. If you're a non-technical founder or just want to build without bottlenecks, try Lovable today for free. Use the promo code EO2YT to get 20% off your first purchase of the Lovable Pro plan.
The good news there is if you have learned how to work with this weird intelligence called humanity, you have everything you need to know to work with this weird intelligence called artificial intelligence. One of the things that cognitive scientists have known for a long time is that human problem-solving and decision-making is improved by a phenomenon called thinking out loud. If you actually get a human being to think out loud about their problem, their decision-making improves, and their problem-solving improves. This is true for yourself. It's true if you're a parent working with a child. It's true if you're a manager working with a junior employee. Having someone just think out loud about how you would solve that problem often leads to a breakthrough. The weird thing about AI is it's true for AI too. This is what's called chain of thought reasoning. And when you get an AI to think out loud, so to speak, it meaningfully improves the outputs of the model.
So how do you do it? It doesn't require some technical wizardry. It requires one additional sentence to whatever prompt you've given it. Give the prompt and then say the following: "Before you respond to my query, please walk me through your thought process step by step." That's chain of thought reasoning. Why does that work? It comes back to the fundamental architecture of large language models. What's happening when a language model is generating a response is it's predicting its next word. A language model does not premeditate a response to you. So, if you say, for example, "Help me write this sales email," it doesn't say, "What's a good sales email? Here it is. Blop." You know, uh, maybe there's a splat sound that we play there, right? Splat. "Here's your email." It's thinking one word at a time, right? So, when you look at Chad GPT or Gemini or many others, and you see kind of the text scrolling, that's not some like clever UX hack. That's not some cutesy design decision. That's literally how the model works. It's thinking one word at a time. But importantly, when it thinks of the next word, it takes your prompt and all of the text that's generated to generate the next word. And then when it's thinking of the next word, it takes your prompt, all that text, and that last word, and it thinks the next word.
So, for example, if you say, "Please help me write an email," almost always a model is going to start by saying, "Absolutely." But then what comes next? "Help me write this email. Absolutely, I'll do it. Dear friend," right? But if instead of saying, "Help me write this email," you say, "Help me write this email. Before you respond to my query, please walk me through your thought process step by step." Now, it knows its job is to walk me through its thought process. How do I write an email? So, it says, "Absolutely, I'll do that." And then instead of saying, "Dear friend, writing the email," it says, "Here's how I think about writing an email. I think about the tone. I think about the audience. I think about the objectives. I think about the context." And then amazingly, it takes all of that reasoning into its process of writing, "Dear friend." Maybe it says, "Now that I've thought about the tone, friend isn't appropriate here. Dear respected colleague," or whatever, right? But the point is, when you ask a model to think out loud or use chain of thought reasoning, it gives the model the opportunity to bake all of its thought process about the task into its own answer. Because the reality is, for a lot of us, we get an output from a language model, and it's a black box. How did it think of why did it think of that? Where did it get that number from? Right? There are all these questions. By asking a model to think out loud, you know the answer to what are all of the assumptions that the model baked into its answer. And now you have the ability again, not only to evaluate the output, but also the thought process behind the output.
Few-shot prompting is another very important technique. It's a foundational technique. You could say it's a predecessor to this kind of modern obsession with context engineering. The idea with few-shot prompting is an AI is an exceptional imitation engine. If you don't give an example, it imitates the internet, but it doesn't do much more than that. And the notion of few-shot prompting is effectively saying, "Here's what a good output looks like to me." And the idea with few-shot prompting is thinking for a moment, "What is a quintessential example of the kind of output I want to receive?" For example, what are my five greatest hits of emails that I'm really proud of, that I think do a good job of conveying my intent or tone or personality or whatever it is? Why not include those emails in my prompt for an email? If you don't give any guidance, it's going to sound like whatever it thinks the average kind of response or the average output should sound like, and most of the time its intuition is wrong.
And then bonus points if you actually give a bad example. If you say, "Please follow this good example and then steer clear of this bad example." These giving real examples is a much better approach than using adjectives. Somebody might say, "Good example is easy, but bad examples hard." It's only hard to the unaugmented person. If you have AI augmentation, which we now all do, you can say to an AI, "I'm trying to few-shot prompt a model. I've got a good example, but I struggle even to think about what a bad example could be. Could you craft the exact opposite of this and tell me why you've done it as a bad example that I could include in my few-shot prompt?" And if you tell it using chain of thought reasoning, "Please walk me through your thought process step by step before you do this," then you'll get a bad example, and you'll get how it's thinking about the bad example. And a lot of times, you actually don't need the bad example. You need the thought process. You go, "Oh, that's true. It's true that my good example is super tight." And the opposite of super tight is verbose. So again, using these tools together, few-shot prompting and chain of thought reasoning, enables you to not only be able to create an example to emulate, but also a really good example to avoid.
The other technique that I think is kind of table stakes for collaborating well with AI is something called reverse prompting, which is basically asking the model to ask you for the information it needs. If you ask a model to write a sales email, it's going to make numbers up. And that can be frustrating to the uninitiated. You go, "Where did it get these sales numbers?" Well, here's my question: Did you give it your sales figures? How would it know? It's put placeholder text in and used its best guess. But if you reverse prompt the model and say at the end of your prompt, you know, "Help me write a sales email. Please walk me through your thought process step by step. Reference this good example and make it sound like that. And before you get started, ask me for any information you need to do a good job." The model will first walk you through its thought process, and then instead of writing the email, it'll say, "I'm going to need the most recent sales figures to be able to write this email." Well, "Can you tell me how much you sold of this skew in Q2 last year?" So, you basically give the model permission to ask you questions. This is part of the core, actually, of the teammate, not technology paradigm. If you're working with a junior employee and you're sending them off on a task, what's one thing you're definitely going to say? "If you have any questions, don't hesitate to ask me." Right? Any good manager. Imagine a manager who says, "Don't ask me any questions." But sadly, AI, in its desire to be a helpful assistant, doesn't want to trouble us humans with questions unless we give it permission to ask them.
Assigning a role is one of the most foundational techniques that you can leverage because it's effectively telling the AI where in its knowledge it should focus. So, very simply, if you say, "You're a teacher," "You're a philosopher," "You're a reporter," "You're a theatrical performer," "Molecular biologist," each of those titles triggers all sorts of deep associations with knowledge on the internet. You start to appreciate why simply giving a role helps, because it starts to tell the AI where in your vast knowledge bank do I want you to draw information and make connections. So, any one of them, I would say, is better than, "Please review this correspondence." But better than just that prompt is saying, "I'd like you to be a professional communications expert. And if you have a favorite professional communications expert, use them. I'd like you to take on the mindset of Dale Carnegie, the author of How to Win Friends and Influence Others. How would Dale Carnegie think about this? How do the principles that Dale Carnegie taught affect and influence and impact this correspondence?"
One of the simplest techniques that we teach at the DH is trying on different constraints. One of the best ways you can solve a problem as a human is by forcing yourself to try on a bunch of different constraints. How would Jerry Seinfeld solve this problem? How would your favorite sushi restaurant solve this problem? How would Amazon solve it? How would Elon Musk? Anytime you make an association, you're colliding different information sources there. The same is true for an AI. An AI is basically making tons of connections through its own neural network. And by giving it a role, you're telling it where do you assume the best source of connection or collision is going to come from?
If I'm going to use AI to role-play a difficult conversation, I typically think about kind of three different chat windows, so to speak. One is a personality profiler. Two is the character of the individual that I need to speak to. And then third is a feedback giver. I want to get objective feedback on the conversation. This I'll show you just how I would have a conversation with Chad GPT to prepare for a difficult conversation in my real life. I'm just going to go into the tough conversation personality profiler and I'm going to say, "Hey, I'd love your help preparing for a conversation I need to have with my sales leader, Jim. He emailed me last night saying that he deserves commission on a deal that I know came through a different channel." And so, I'm just kind of giving a little bit of background. I will just upload that to the personality profiler. And what this one's been taught to do is I'm going to start with step one of the process, gather intelligence about the character and the scene. Right? I'm just going to look at the questions here, and I'm going to I'm going to use my voice to answer them because it's a lot easier than using my fingers. Okay, first question: How would I describe Jim's communication style? Um, he's quite direct and confrontational. He's kind of typical East Coaster, sarcastic. Well, I know that it came from our through our social team. There was a cold LinkedIn campaign that they ran, and I know the CTO actually responded to that campaign. So, and then best-case outcome of this conversation? One, I mean, I'd like for Jim to kind of back down. I mean, like near-term, I want Jim to back down and agree that the social team gets the commission. So, it's kind of recapping its understanding, and then it's giving me these instructions. It's saying to me, the user, "Copy paste the following instructions into a new Chad GPT window." So I've just copied those instructions. So now what I'm going to do is I'm opening Chad GPT, and I'm just going to go and first thing I'm going to do is start a new conversation with Chad GBPT, and I'm just going to paste those instructions in, and I'm just going to paste them in. And what it should do is hopefully say, "Got it. I'm Jim." So, I'm just going to go in and I'm going to start it. We're going to see how this goes. You guys ready? "Hey, Jim. Um, do you have a second?" "Sure thing. I've always got a moment for a good chat. What's on your mind?" "Uh, yeah. I just wanted to clarify. It seems like um the commission that you emailed me about last night from the telco. I'm I'm afraid you might be mistaken there." "Mistaken, huh? Well, I'm definitely open to hearing the details. If there's something I missed, let's lay it out on the table and figure it out." "To be honest with you, I was a little bit caught off guard when you uh when you sent your email last night." "All right, I hear you. I mean, if the social team really did the groundwork to bring that lead in, then I definitely want to make sure credit goes where it belongs. So, if I hit X, I just I end that conversation."
What's interesting to me is I kind of feel like that wasn't totally Jim. And either either that went well, or you know, I don't know what I did well. But one of the beautiful things about voice mode is now I've got the whole transcript here. I can just screenshot it. Right? So I can take screenshots of this conversation. I can put them into a new GPT window and I can get feedback and I can say, "Hey, tell me what I did well. Tell me what I did poorly." Or I can go back into my conversation history here, and I could say, "Hey, I just had my first conversation with this version of Jim, but I feel like he kind of he was too agreeable to be honest with you, and so I'd like you to incorporate a little bit more edge into the instructions to the character that's going to be playing Jim. Would you mind to update your instruction set?" Right? Because ultimately, what you want is you want to have a realistic experience of the conversation. So, so now what I'll do is I'm actually going to open a new conversation, and I'm going to paste the updated instruction set, and let's see. Let's see what kind of Jim I got here. "Big voice, bigger results. I fight for what's mine. I don't appreciate being second-guessed." So, now I'm going to try again. Let's see. "Hey, Jim. Do you have a minute to chat?" "Hey, absolutely. I've always got a minute." "Oh, that one. Yeah, I I figured that might come up. I I definitely think I had a hand in that deal, so I'm curious to hear what you have to say about it." "Yeah, really appreciate it. I mean, you what you and the sales team are doing is awesome. Totally appreciate you. Um, that being said, I actually feel that attribution is really important that we get airtight here. And the truth is, I I am fairly confident that deal came through the social team." "All right, I get where you're coming from. I definitely appreciate you wanting to clear the air and to keep things fair. So, what kind of evidence do you have that it came through the social team?" "Well, so I've been through kind of two windows. One is profiling my conversation partner, and then the second is having the conversation."
All right. So, what I'm going to do here is I've opened my tough conversation greater GPT, and I'm basically just going to upload photos of that last conversation. And what this GPT has been trained to do is basically give evaluate my conversation and then let me know how it went. "Thanks for sharing the full transcript. My first step is to understand the objective. Step four, here's your grade. You got a 78 out of 100. You succeeded in preserving trust and resolving the immediate issue." So, I can take all of these. I can even say, "Hey, would you give me a quick one-pager of a handful of talking points that I should probably make sure not to forget in the order in which they're likely to emerge in this conversation based on the feedback you've given me?" The AI will actually give me a really short kind of at-a-glance conversation guide that I can leverage if I want to try again. Right? "Here's a one-pager. So, these are all great points. Now, I can bring them into the conversation. I actually I'd probably do this a couple times before having a real conversation with Jim. But the point is, historically, the only time I get feedback is after I have the real conversation with Jim. This is the first time in history, and maybe I can get a friend to kind of go over talking points with me. But unless they're really close to Jim, or unless they're, you know, particularly imaginative, and unless they're deeply knowledgeable of a bunch of feedback frameworks, they fall short of really preparing me in context for this specific situation in the specific conversation I need to have in a way that AI is able to help me. You can use this for any difficult conversation, whether it's a performance review, a salary negotiation, difficult feedback. It's a great way to basically get a flight simulator for a difficult conversation."
The people who are the best users of AI are not coders. They're coaches. They aren't developers or software engineers. They're teachers and mentors and people who have learned to get exceptional output out of other intelligences. And so, where could AI go? Well, it's really a function of who can get unleashed. Right now, the primary limitation is the limits of human imagination. And as we unleash and ignite and spark more human imaginations, the kinds of applications that are possible are they're unthinkable, not because they're technologically impossible, but because they never occur to us personally.
One of my favorite quotes is a Nobel Prize-winning economist named Thomas Schelling. He said, "No matter how heroic a man's imagination, he could never think of that which would not occur to him." If you take as a premise that the imagination space is a function of what would occur to various individuals, then as we equip different individuals, what we can imagine collectively expands. In innovation studies, it has been called the adjacent possible for a long time. What is possible is just adjacent to what is. And as we increase adoption and increase fluency and competency and increasingly mastery of AI collaboration, then we're increasing the adjacent possible. And it's really important that you exercise through implementing some of the things you hear. And perhaps the most important thing you could do with this video is actually hit stop and do something that's already blown your mind.