Transcription
Today we're going to learn about prompting from librarians, and I'm going to break down, uh, this report and also tie it into vibe coding, just ways that we can improve our prompts and a framework for giving better instructions to LLMs.
What do I have in front of me? I have, um, a report, uh, from the university libraries and learning sciences faculty and staff publications. Uh, that's a mouthful. Um, but what it is, is a prompt engineering guide, and we're going to talk about the clear path today. Now, it's important to note this was written almost two years ago. So, I'm going to call out some things in here that are maybe a bit dated. Um, but shout out University of New Mexico, uh, for dropping knowledge bombs on us courtesy, uh, of 2023.
So, this is, um, from the Journal of Academic Librarianship. Uh, so, we're learning from librarians. If you're a librarian and you're watching, thank you. I love libraries. I remember I have a lot of fond memories from libraries as a kid. Um, and we're learning about prompt engineering from librarians. So, Leo Lo, you're the man. Thank you for your service.
So, we're going to talk about the clear framework. And the clear framework is concise, logical, explicit, adaptive, and reflective. These are all things I'm a huge proponent of. So, let's just get that out of the way. These are all really good, uh, adjectives. We're going to look into them and understand them a bit more. Kind of abstract here and prelude. Hey, AI is advancing really fast. Models are altering the information landscape. Uh, we need some fresh perspectives. And to address the need, uh, our author here developed a clear framework for prompt engineering, a concise and user-friendly method tailored to optimize interactions with generative models, particularly for beginners. And, you know, one thing I really like here that we're going to carry over into our prompting advice is that this is simple and it's straightforward. And I think, uh, a lot of us, especially people who like to talk about coding and whatnot, tend to overcomplicate things. It doesn't have to be complicated. It can be a bit more simple.
So, we're going to talk about this clear framework, and I'm going to just apply it directly to vibe coding, to prompting AI, everything we could need.
So, the clear framework for prompt engineering. What is prompt engineering? In essence, it is the art of fine-tuning the questions or commands provided to AI models in order to optimize their performance and guarantee that they produce the desired result. Now, I have a video out on context engineering. Engineering, and it's really useful, and I think it follows from this video, so I'm going to link to that as well. But context engineering is a newer term. It wasn't really around in 2023. And so context engineering is like what you attach to your prompt, how you provide examples and documentation and images. Prompt engineering is much more just like how you phrase the prompt. And that's important too. I think prompt engineering is a subset of context engineering. Um, but don't worry too much about, you know, confusing terms. Let's just think about prompts, and these, uh, these characteristics are applicable. Um, so what else do we have here? The purpose of the clear framework is to provide a standard method for composing effective queries for language models. So I'm going to help you today not just with vibe coding or prompting AI for writing code, but just with everything you could prompt AI for.
And the analogy I like to use here, I think it's a pretty direct analogy before we get into the framework, is like prompting. It's like writing a, a Google search. Google search. Um, every time you write a Google search, you have a way in your brain of taking a question and turning it into something that works with Google. So, when you're prompting, you really just have to think, okay, what works with AI? How do I take that thing in my brain and turn it into something that works with AI? Now, AI is pretty new, so you might not know how to do that. That's what we're going to talk about today. But, you know, like on a Google search, you might just put like Reddit at the end, and that might work. But, what you're doing is something very specific. You're filtering for results from Reddit. And so that's a strategy. We're going to talk about some AI strategies.
First, concise, brevity, and clarity in prompts. I love that. You know, you have to really think about AI like, um, a colleague or someone you work with or a friend. If you're sending somebody a text message, I've used this analogy before, you can't send them 2,000 words. Now, I wouldn't know because I don't have any friends, but if you do have friends and you send them a super long text, they're not going to read it. The same thing is kind of true of AI. If you send a lot of information that doesn't matter, AI is more likely to get confused or, uh, return incorrect results because it misinterprets what you're asking. And so, the first component of the clear framework emphasizes the importance of conciseness in crafting prompts. A concise prompt removes superfluous information, allowing AI language models to focus on the most important aspects of the task. This is like really important stuff. How does AI work? Under the hood, AI is just a statistical machine. It's taking tokens or like combinations of letters that you put into the model and then predicting, basically, outputs based on the data that's trained on. So if you give a bunch of like kind of dilly-dally, you know, very verbose language to an AI model, it's actually going to influence the output. And that's something that we have to recognize. Models are logical. And so you have to boil it down as if you were instructing a very logical person on how to complete a task. So we're going to remove superfluous information and eliminate conflicting or incorrect information. That's important. Um, what else do we have going on here? Uh, so they call out that this long, uh, irrelevant content may not meet the user's needs or expectations when the model responds.
And so some examples here: Explain the process of photosynthesis and its significance instead of, "Can you provide me with a detailed explanation of the process?" So, right, we have like this whole preamble here that we could just cross out and really get to the core, which is the process of photosynthesis. Instead of requesting, "Please provide me with an extensive discussion on the factors that contributed to the economic growth of China during the last few decades," use a concise prompt like, "Identify factors behind China's recent economic growth." This is a, this is a better example here, right? This is just like kind of, um, very long. And so, you know, tokens matter in aggregate. Like the, the, the shorter the input that you can give to get, uh, a more targeted output, the better off you're going to be. So we need to to pair down our requests. Sometimes that might mean doing legwork. So if we tie this back to like writing code with AI, if you can do some debugging to understand, uh, what's going on with your model, you're going to be better off.
Let's actually look at a really targeted example of this. So you can think about prompt engineering or even context engineering as bridging the gap between what AI can do and maybe what it can't do if you're trying to solve a problem. So say I'm trying to write code, uh, building with a Replit, and, um, there's a problem: user information is not displaying, and we have an agent or an AI system that can help us solve this problem, but we don't know the cause, or the agent doesn't know the cause. It can't have, it doesn't have access to these things. Maybe a table in your database doesn't exist. Now, maybe you know this, maybe you don't, but the agent definitely doesn't know this. And so your job when you're engineering, prompt engineering, context engineering, all these things, is to bridge the gap between the problem and the solution. And the way that you do that is by using context and using, um, logic to help AI think through the problem. "It looks like fields that require user information are not loading. Is there a problem with the database?" So, I'm pointing our agent in the right direction. And this, this prompt is very targeted. And it logically connects two things: the issue and a potential solution. And that logical connection is what helps AI solve the problem. And part of that is being concise. And part of it is drawing a logical connection. We're going to talk about that more.
So, the very next, uh, the very next section here: logical, structured, and coherent prompts. So, we're being concise. We're pairing down requests. And now we're being logical about our prompts. And this is going to tie into what I just talked about. The second component emphasizes the significance of sustaining a logical flow and order of ideas within a prompt. So I just put out a video on, uh, context rot, which goes over a paper that, uh, Chroma released. And basically, what that paper shows is that the way we prompt matters. Couple things. First, AI is better at lexical matching than non-lexical. What do I mean by that? Explicit instructions always trump non-explicit instructions. What's an example of something that's non-explicit? Basically, something that you'd have to infer based on, uh, based on circumstance, versus me directly saying XYZ. So in the example I just showed you, I said, "The issue is that our user information is not showing, and there might be a problem with the database." We are logically connecting those two things. And so if we can make direct instructions for our AI systems, that's much better than indirect or illogical or potentially, uh, having to, to jump from one conclusion to another, because AI isn't that great at that.
So to create a logical prompt, ensure that the provided information follows in a natural progression and the relationships between concepts are evident. That is the, the core of it, right? The relationships between concepts are evident. "List the steps to write a research paper, beginning with selecting a topic and ending with proofreading the final draft." "Describe the steps in the scientific method, starting with forming a hypothesis and ending with drawing conclusions." Those are good examples. If we're going to tie this back to like vibe coding or writing code with AI, uh, we'd think about taking a solution and tying it to our problem or what our app is going to do. You know, "Hey, I want to build an app that helps me, um, you know, like analyze information for my team." Well, that's a little ambiguous. "Help me build a form submission app that allows users to submit their data and flows through to a Kanban board where my team can analyze it in aggregate." So, we're being very specific. We're drawing a connection from the user interface to the, uh, analytics and kind of like the way we're interacting with the app. Um, and so the more logical we can make our prompts, the better off we'll be. I think a great, great opportunity for being logical is solving problems and is fixing erroneous output.
Okay. Third, explicit clear output specifications. It's kind of what I was getting to as well. These all tend to overlap, right? Explicit prompts provide precise instructions regarding the desired output format, content, or scope. Include specifics about the type of information you seek and how it should be conveyed to make a prompt explicit. Amazing explanation here is like error logs. If there are error logs that AI doesn't see, you need to provide the errors or like what, what's going wrong for to in order to write code. Um, if we're just talking about prompting more generally, uh, yeah, you have to be explicit about exactly what you want because AI is kind of like a genie in the bottle. You know, if you don't tell it something, it'll like make something up. It'll give you, um, non, uh, optimal responses. And so if we're talking about, right, vibe coding, or writing code with AI, error logs, images that display, you know, poorly formatted data, um, code snippets or examples, these are all great ways to get to convey explicitly exactly what you want. So, you know, another example I had down here: "Hey, instead of 'add authentication,' 'implement authentication for my admin page so only I can access it.'" We're telling AI exactly what we want.
So, uh, we have, um, concise, logical, explicit. Next, adaptive flexibility and customization in prompts. What are we talking about? Adaptive prompts emphasize the significance of of adaptability in prompt engineering. Adaptability entails experimenting. I say this all the time. It's almost like this guy read my mind, except I didn't even know about any of this in 2023. So what am I talking about? With various prompts, formulations, phrasings, and temperature settings in order to establish a balance between creativity and concentration. It also necessitates adapting the AI model's responses to the requirements of each mission. I love that he calls it a mission. This is great. So, one of the reasons I love this one is because I think sometimes as, uh, architects or people trying to build things, we want like things a specific, specific way. But that's not always the best idea because sometimes we actually want AI to surprise us. And that's one thing that I notice that vibe coders do a lot, or people that are building with AI. They'll like generate these PRDs and then they'll be like, "Build me exactly this thing." And sometimes if you don't give AI any like wiggle room to experiment, you lock yourself into a sub-optimal result. And so I think kind of what, uh, the author is saying here, this adaptive approach, is that we have to learn how to interact with AI. We have to learn how to interact with AI. But we also, right, uh, have to give AI a chance to surprise. And that's because of like the whole non-determinism thing. Sometimes AI gives us something good, sometimes it gives us something bad. We want to allow AI, we want to allow the upside, right? Something really great. Um, and so I love the idea of experimenting because the other fact of the matter is that, um, no one is an expert. And you might think, "Hey, Matt, you're on this YouTube video. Aren't you an expert?" I'm absolutely not an expert. I've only been doing this for, you know, a couple years. And it sounds like, uh, Leo here has been doing it for longer than I have. So, no one is an expert. You can discover things that work well for you. So, that's important.
Um, now let's take a look at these examples. Let's see what Leo's got for us. "Discuss the impact of social media on mental health." Elicits responses that are too general. Consider a more focused and adaptable response. This is another really great approach. So sometimes, right, what I'll do is I'll start high-level and then I'll drill down because like I'm, you know, candidly, I can be like a little lazy sometimes. So I'll be like, uh, let's say, "Drill down." So we'll start high-level and then I'll draw an arrow just for, "Okay, that didn't, that didn't draw an arrow at all." Um, and then we'll drill down. And what do I mean? And I'll say like, "Oh, fix this. The database is broken. Fix it." If that works, great. If it doesn't work, okay, why is the database broken? We need to understand that relationship. Uh, so if asking, "What are some ways to conserve water?" leads to generic responses, try a more targeted approach. That's like exactly the same thing. "Describe history of computers." Too much information. Use a more specific and adaptive prompt. And, you know, like if we're tying this into vibe coding, sometimes we try initial prompts and they just don't work. And what I found is that like sometimes it's better to just say like, "Hey, that didn't work. You know what? I'm just going to scrap it and try another prompt that gets me closer to where I want to be." Because that initial prompt kind of kicks off a trajectory of what your app's going to look like. And it can be really good to to to spend some time tuning that initial prompt, tuning that initial prompt, or being adaptive with how we prompt. I don't know if y'all do this sometimes, but sometimes I'm talking and I don't even know what I'm saying. It's just like my brain's all over the place. So, forgive me.
Okay, we got our last, our last characteristic here. So, um, concise. I can't remember anything today. Concise, logical, explicit, adaptive, reflective. Reflective. How? Tell me more. Continuous evaluation improvements of prompt. So, is this talking about the model? No, it's talking about, um, you adopting a reflective perspective. Reflective perspective allows users to evaluate the performance of their AI model based on user feedback and their own assessments. Again, I love this. This is very similar to adaptive, but we're saying, "After receiving AI-generated content on the benefit of a plant-based diet, I love chicken. This, you know, if you're a vegetarian, I respect it, but not for me." Evaluate the response's accuracy, relevance, and completeness. Excellent. "After acquiring an AI-generated list of strategies for effective time management, evaluate the relevance and applicability of each strategy." Also excellent. So what we're saying here is basically, uh, check the model. Three exclamation points. That's important. If you ask AI to implement authentication or user management or straight billing, you have to check and test the work out, the workflows. You have to to evaluate the accuracy, relevancy, and completeness. And it's important to then take what you saw and carry it over into the next prompt you give. So you have to be reflective in the sense that you're saying, I prompted AI and I tried this approach. Did it work? If it didn't work, I'm going to try something different next time, and I'm going to start to make notes of these things.
Now, there's another kind of call out here that I think is, uh, important. There's this thing called chain of thought prompting. There's also, um, just some other research around asking models to reflect on things or think through things. So chain of thought is explicitly asking a model to think through steps. A lot of times AI agents, so like Replit, will handle that for you. But it's a good, uh, characteristic to keep in mind if you're thinking about how to prompt AI better. Sometimes saying, "Hey, think through this process," or prompting the model to be reflective. "Help me evaluate several options for X. Which of these options are the best option?" Sometimes even saying, "Think from the perspective of," right? You know, we all know how AI can kind of tend towards agreeing with us or can be overly agreeable. Say, "Think about this thing as if you were arguing from the opposite perspective, and then help me understand how I could go forward." So there are all sorts of ways to get models to be a bit more reflective, or, um, even to have help models see an opposite perspective that can be useful, and those are some other characteristics that I think can be really helpful.
And so this framework provides, uh, academic librarians and also me and educators. There we go. We're in here somehow. Uh, with an easy-to-remember approach to focus on prompt engineering. What do we got? Concise, logical, explicit, adaptive, and reflective prompts. These are all great prompts for vibe coding. They're great prompts for librarians, too. Um, but I don't know much about librarians, and I also know questionable amounts about vibe coding. Um, but I think these five pieces of advice are really great. So the next time that you're writing a prompt, think: Is it concise? Is it logical? Um, and is it explicit? And then how can you adapt to the results from that prompt? And how can you reflect on the outputs and use that to feed into your next prompt? And I think you'll get better results, uh, if you follow this framework than if you follow none at all. So again, I'm Matt. Until next time, peace.