Transcription
For a long time, I have wondered what the difference is between prompt files, custom instructions, and custom agents in Visual Studio Code. And it's a question I hear a lot. So, in this video, we're going to dive in. We're going to take a look at all of those things, how they work together to completely understand when and where to use them. Are you ready? Let's go.
Before we actually look at custom instructions or prompt files or custom agents, it's really important for you to understand how the agent system prompt works in Visual Studio Code. And this is important because you're going to need to know where in fact these instructions get inserted into that prompt. So, let's run through that just real quick.
So, if we were to type some sort of a message, just say hello world here and send it to the chat, it gets sent. But let's take a look at what actually happens behind the scenes here. Behind the scenes, Copilot composes a prompt and it starts with a system prompt like this here. And the system prompt starts off with some core identity and global rules. And this is just very generic stuff. In fact, I think there's only two or three lines. It's like you are an intelligent AI coding assistant. Kind of this just generic identity. And then underneath that there's general instructions. And these instructions can vary by model because the models have various quirks. So for instance, if a model is very aggressive about writing the code out to the chat when it should just be writing it to the file, there might be an instruction never print out code blocks with file changes. And then underneath that there's tool use instructions. And these are instructions for the model about how to use the tools that are included in Copilot like um the edit tool or the terminal tool or the to-do list tool, any of the built-in tools. And then underneath that, there's some output format instructions that just tell the agent how to format the output for tokenization in the chat so that things like the little file pills that you'll see in Visual Studio Code actually show up correctly.
Now the next thing that happens is a user prompt gets added to the message that's sent to the model. It hasn't been sent yet. The user message contains the in environment info. So information about the user's operating system, etc. And it also includes workspace info. And it literally looks like this. It's kind of your project structure in text format. Project name, folders, files. And then another user prompt gets added to the message which has still has not yet been sent and that contains context info like the current date and time list of open terminals that you have. It also contains any files that you have added to the chat. Now, in our case, we didn't have any files added to the message, but if we were to add files like that, then they would show up here, right here into this editor context. And then finally, we have the hello world. And then all of this gets sent up to the model and the model responds with this assistant message. And all of these things together are the context window. So the context window is just being built up here. We have one and then another one gets added and then uh another one gets added here and then all of this gets sent and then the response comes back and then this gets added and now this is the whole context window here right all of this and if we were to add another user message this would just get added here and would become part of the context.
Okay, so now you understand what actually happens when you send a prompt. Now the question is what are custom instructions, prompt files, and custom agents and where do they actually go in this prompt? So let's take a look at that. It's probably best if we start with something like custom instructions.
So the most canonical use for custom instructions is to contain highlevel information about your project that might help the model do a better job giving you answers. So for instance in Visual Studio Code there if you click the gear here we can generate chat instructions and this will do exactly what I just said. It will create an instructions file that has the highle information your project architecture any patterns you can use. You can actually go and read this prompt here so you can see exactly what it's being asked to write. But we recommend this for every project. And this is probably the most common use case for custom instructions. Once we've generated these custom instructions, if we were to send another message, you'll see when we do that, the custom instructions automatically get passed right there. There they are. So this file is being passed as part of the context. So the question is where exactly does that show up in the prompt? So in our diagram here, custom instructions should show up right here in the system prompt and they're actually added right here. So they are the last thing in the agent system prompt. And it should be noted that the copilot instructions will always be the last thing in the agent system prompt because you can create as many instruction files as you like.
So let me give you an example. This site is called Awesome Copilot and it is full of community contributed prompt files, custom instructions, and custom agents. It is a treasure trove of custom instructions that you can look at for inspiration. So, we could go here and find some instructions that we like. So, here's a table of all the different ones that are available. So let's go down here and let's pick maybe um NextJS best practices for LLM. So if we click install, this will install the file in Visual Studio Code. We can go ahead and accept it. It asks us where we want to put it. We can put it in the user data folder which makes it globally available or we can put it in the github/instructions folder which means it will only be available in this project which is exactly what I'm going to do. And uh now if you look we'll see we have an instructions folder and a next.js instructions inside. And if we were to pass another prompt here now you probably guess what's going to happen. Both of those files actually get passed. And just to clarify, remember that the nextjs instructions will always come before the co-pilot instructions file. That always comes last. So at a high level, that's custom instructions.
Now, let's talk about prompt files. Prompt files are reusable prompts that you can define and then use right in the chat. Now, why would you do this? Well, let me give you an example. If we want to define a prompt file, we can go to configure prompt files. And you can see that I've got a bunch of different prompt files here. So, let's go ahead and look at this one here, which is called remember. So, I'm going to look at the remember prompt. And in the remember prompt, this is a prompt that I have that just builds up a memory file. So, I can tell the AI at any time to just remember something and it will do that. Now, you can see up here in the front matter that I can specify the agent down here. We're just going to stick with the agent. Um, but then we can also let's add a description. But then the other very powerful thing here is that we can also add a model. And you can see we get IntelliSense for all of the models that we have available. Many of these are on open router. Some of them are built in. I'm going to use a small model for this so that I don't waste premium requests. And what this means is when I use this prompt file, it will automatically move us over to the correct model. So let me show you what I mean. So let's say the model keeps making the same mistake of trying to use a use effect on server components which you cannot do in Nex.js and so we just want it to remember not to do that. So we can use that prompt file with just the slash here and then paste in our message. And when we send this, you'll notice that we're on Claude Opus 45, which is a premium model at 3x. But when we send it, it actually gets sent with GPT41 because that's what we specified in the prompt file right here, gpt41. And now it's created an instructions file for us. And this instructions file will be added automatically to every single request. So you can see here how I'm kind of starting to build up workflows using actually both prompt files and custom instructions.
Now let's go back to our diagram and let's take a look at where actually in the system prompt these prompt files actually show up. So the answer is they don't not in the system prompt. They actually show up down here in the user prompt. So in the user prompt what happens is these prompt files their contents get added right here at the very top. So even before the context info, we'll have prompt files. And then we're going to have just the contents of the prompt file that was used. And then what happens is down here in this message, it says follow the instructions in and then it points back up to the contents of that prompt file, but it actually uses a special syntax to do that. It's kind of it's pointing to it by name even though it's actually included in the user prompt. And then after that it has your system message which would be use effect can only be used uh in client components. Right? So that's where prompt files come in. And so if we have a lot of messages, they may be way way way down in the list, right? Maybe way down here. Each time we use a prompt file, it gets it's part of the user prompt, not part of the system prompt.
Now, the question that you're going to ask is does the placement of the prompt within the message hierarchy actually matter? And the answer to that is I don't know. However, context rot is a real thing. And the basic idea here is that as the context window grows and gets longer, the performance of the model degrades. Now, this has gotten better over time. As the article dives into here, it's improved. But you can still see here that as the context window grows, for example, a 32,000 token prompt, accuracy drops dramatically. Even Claude 35 Sonnet goes from 88% to 30% accuracy. So it's important to remember that as your prompt is growing here. So if we have a system prompt and a user prompt and a user and then an assistant message and then a user prompt and then we have another assistant message because we haven't started a new chat. So we just keep going and going and going. It doesn't really matter if you're using custom instructions or prompt files. The performance or the accuracy of the model is just going to degrade. And this is one of the reasons that the token windows or the context windows in VS code are limited at a certain point and that is to maintain performance. So it's hard to answer the question would you be more accurate to pass your instructions as a prompt file or a custom instruction. The best thing to do is instead of worrying about the positioning just use them as they are designed which is to help you compose workflows and not worry about their position in the prompt.
Now, let's talk about one last thing, and that's custom agents. Custom agents used to be called custom modes, and I built one a while back that was called beast mode that was designed to help GPT4.1 perform better. And the idea here is that you can pass instructions to sort of override or augment the default agent behavior. So, let's take a look at these. What I want to look at is one of the ones that we're now shipping in Visual Studio Code, which is the plan mode. So, you can actually click on configure custom agents and you can look at our built-in plan mode here. So, let's take a look at the plan mode. You can see that it the name and the description and then there's these tools and then there's these things called handoffs which we'll get to in just a second. And then you can see that it's very much like we're writing an agent prompt. This is very different from custom instructions, right? If we look at the custom instructions, this is different. It's just giving it information. This is giving it an identity. So, it's very much like an agent system prompt. And then it's going to go through and it uses a workflow to start a planning process with the user. The first step is to gather context and research, then present a concise plan for iteration, and then handle user feedback. And then uh it will give you the option to either implement the plan or to put the plan write it out to the editor and it does that via handoffs.
So let's actually run this and take a look at what this actually looks like in action. So I'm going to pick a model that's a little bit better at planning here. I'm using our built-in plan mode. So let's do something that is always done in these demos. We're just going to add dark mode to an app. I don't know why that's the the prompt that's always used, but it's because it's visual and it's because something we can see. So, we're going to let Haiku go through and work through this plan. You can see this and then we'll come back and talk about what is actually happening here.
Okay, so we're using this new plan agent and we've sent this prompt. What is actually happening behind the scenes? Let's go back and take a look at our diagram. So, what happens when you use a custom agent is this. it actually gets added to the system prompt here. So, let's make some more room. And the custom agent is always added here below your custom instructions. So, that's the order in the system prompt. The custom instructions will be added to the end of the system prompt and then the custom agent instructions are added to the end of the system prompts. They are the last thing in the agent system prompt.
So now let's take a look at how we can use custom instructions, prompt files, and custom agents to compose Agentic workflows because that's what they are. There isn't really a right or wrong way to use them. They're just building blocks for composing your own workflows, but that doesn't really make any sense in the abstract. It's going to help you if you see how someone else has done it. So what I want to do now is show you how I compose workflows. So what I'm going to do is I'm going to use a prompt file for planning. I have a custom planning prompt file and then I'm going to ask it to refactor the UI of this application to be more clean and modern. And you'll notice the first thing that it does is it switches us over to Opus45 because that's what I've defined in the prompt file. We can actually take a look at that. If we go here, here's my prompt file. You can see I've defined the model and then my prompt file is for planning is very similar to the built-in agent but my prompt file actually instructs the agent to work in the concept of a branch. So the plan is really defining one PR or one branch and it's just defining the highle steps. So, it's going to do things that the built-in plan uh custom agent does, like researching the codebase and asking me questions, but the plan format that it spits out is a little bit different because this is just one of the building blocks and it'll make more sense here in a second.
Okay, so planning mode is done and we have one file here that's been created and I have it create the file automatically. It does have some questions, but they're they're mild. Not anything that I really need to address. But you can see it just breaks things down into steps. Step one, do this. Step two, do this. The idea is that each one of these steps is a commit. And commits should be small and testable. And commits will build up and create a single PR that we can then submit. We don't have any code though. So the next step is to actually get code. So let's go ahead and keep this. And because we don't want context rot, let's start a new chat session here. Clear the context. So now what I'm going to do is something a little bit unique. Instead of actually implementing the plan, I am going to have it generate a document based on that plan. And then I'm just going to pass the plan file here. So in this case, I'm using a generate prompt and passing in the plan that we just created. And I'll explain what it's doing. So the generate prompt takes the plan and then it writes all of the code required to implement this plan, but it doesn't write that code in the project. It actually writes it in a markdown file step by step. And the reason why I'm doing this is because I'm trying to maximize my premium model usage. I've used Claude Opus 45 twice now. It is a 3x multiplier. the six premium requests. I want to make sure that I'm getting the most bang for my buck. So, I'm actually going to use a smaller model to implement and a bigger model to write the code. The smaller model will implement it, but the bigger model writes it. So, in just a second, we'll take a look at this implementation plan, and you can see exactly how this works. It's not complicated. The implementation plan that gets generated is long. It's very long. This one's almost 2,000 lines long, but you can see that every single piece of code that is needed to accomplish this job is actually in this file. But what's more uh important here is that they're all broken up into steps. And each step has a checkbox here.
So now that we have that, we're going to clear the context window again, making the best use of our context. And now we're going to use a custom agent, which is just called implement. And all we have to do is then pass in the implementation plan here like this uh and give it a simple prompt and then we just send it. And you can see it automatically moves to the VS code prime or raptor prime which is a five mini variant and it's the model that I like to use for implementation. So we'll go ahead and send that off. Now, as this smaller model works on this document, it's going to implement what's in the document just verbatim. It isn't actually writing any code. It's just implementing the code that the large model wrote. And this strategy lets you sort of oneshot with a huge model and then implement and iterate with a small free model. And this model will keep going until it's completed whatever step it's on in the implementation plan. And then it will stop and it will return control to me so that I can test make sure that I like it and then I will just stage and commit. And then I will just redo what I did. Use the implement agent pass the implementation document and it will just pick up with step two and continue. And I will just iterate with it like that until we get to the end of the implementation and everything's working. And then I have a pull request that I feel really good about.
Okay. So now you have a really good understanding of how things work behind the scenes for the agent in VS Code. You understand the agent system prompt and how the user prompts are added on and where custom instructions and prompt files actually go in that prompt. Go forth and create AI workflows that work for you. I'll put links to mine below and check out the awesome co-pilot repo where you'll find my workflows plus tons of other prompt files, instructions, and custom agents that you can use today. And as always, happy coding.