Transcription
Welcome to the first part of everybody. Wait, wait, wait. [laughter] Are you ready? Welcome to context engineering. Uh, and how to do it properly, how to do it with all the new things you saw on the keynote. Trying to bring it all together. And that's not just for you to assign more of your daily chores to the AI, but to generally getting these things right means you can run longer, more complex tasks with the LM.
I work on VS Code. So the first round is going to be checking out how we in the VS Code team are doing that. There's some great music playing over there. This is amazing. Love this stage. Um, that's the intro music for me. And then later on we're going to do some engineering context in action and go beyond to show some more hands-on examples.
So to understand context engineering, you need to understand what the agent sees out of the box and mostly understand it's your codebase. So what does it actually mean to look at your codebase and what else does it see? In VS Code, the agent that you know and love is copilot. It understands not just your codebased context with the repository. It can find code code using a vector-based semantic search index, which means it can run much broader queries than just specific files and function names. But also understand if you're looking for authentication, if that's named login, it can still find it. And that's one of the key principles of what makes this agentic flow work so well.
It can also run terminal commands and look for the final execution which you all have seen in many of your agent flows. But it can also look for errors in these agentic flows. Tasks. How many are FBS code set up with tasks? You have like quick build task, linting task and all the other stuff. It's a key ingredient for the agent because the agent understands which tasks are running in the background within VS Code and can react and look them up as it makes changes. If you have a task running that watches your code, rebuilds and potentially bails on errors, the copilot agent loop can be aware of that and check it after each turn. We actually use that in VS Code. I'm going to show that.
We also have symbol and pattern search and that's run by the inapp language intelligence built into VS Code. What this means for you is that you should make sure that every language you use in your project has the proper extensions and language servers set up. So if you don't see red squiggles or you see too many red squiggles because your linting isn't set up properly, that confuses your humans and that confuses the agents as well.
Next up, we have tests. So, we you can set up tests to run within VS Code. You can set them up to run in terminal, but VS Code has a built-in test runner that as well is exposed to the agent. It cannot only run test suites, it can run partially uh partial tests and it can has access to any failed tests as well. So do the investment setting up tests and all these things properly and the agent and your humans using VS Code will both benefit.
Lastly, problems and that's connected again to the language intelligence. Anything that has linting errors is again exposed to the agent and that's often live after it makes edits. It gets that as feedback. If you made an edit and the function doesn't exist, linting will call that out and the agent is immediately aware of that. And that's not something you easily get on a CLI. But in VS Code, because you can type code and get these red squiggles, the agent gets the same benefit. Again, make sure that you have the right extensions installed and you even do more linting. A classic one that I recommend is accessibility linting. There's multiple accessibility extensions that would actually put red squiggles under anything that doesn't comply with accessibility rules and that's a just language one. So you can even expand on top of that. So all of the ones that are marked you have to make sure you have them set up in VS Code and to benefit the agent and getting that out of the box context.
Just touching on the code search updates we've been doing. There's a new embeddings model and these these are all from a blog post that came out last month with better retrieval quality where we see higher acceptance rates and faster throughput. It's a custom model. Previously we've been using GPT models for that and this is real world benefits that you see that things are just working faster or more reliable and that's really helps the agent to do less turns to find the right code but also helps you that you can be have more ambiguity in your request and the agent still discovers the right parts of the code to make the changes.
Okay, let's dive into actual VS codebase. So this is github.com/microsvs code. If you want to see what I'm showing, just open up the VS code repo as all of it is out in the open. So that is the VS code repo and even we as a team have been exploring the space like what is context engineering. There's no hard and fast rules. There's no specific files to add. It's more the technique of how do you optimize context and compress it enough and make it available enough for the agent to pull in as needed. And the best starting point and that's what I recommend everybody is copilot instructions.
So copilot instructions you can also use agents.mmd as a cross agent system is really at its core a mini map of your codebase. You tell the agent where things are generally at. So it spends less time looking at code and trying to figure out where where it needs to make changes. You point out how to run it, how to find related code, which tools work best in this codebase is the general rule. How to do validation is specific linting tools it should run is how it should run the tests. In this case, we use tasks. So we pointed to the task output. So anytime it makes changes, it should check the tasks. Again, pointing back to what every developer is doing on the VS code team and doing as well working. Yep. So all of that is now in the repo and helps every developer, every contributor working on this repository out of box. These rules are included out um by default.
Next up we have these domain specific instructions. There are instructions that are included by the agent looking at description and pulling it in as needed. We are describing here little micro concepts that are shared across the codebase and often times based on what the agent gets wrong. We have for example an observable pattern that is pretty uh specific to our codebase and where the agent keeps coming up with new APIs or new patterns to use this the system that don't exist. So we just want to guide it towards we want to grow the adoption of observables in a right way and giving it this little piece of information is enough so it gets it correct more often. So anytime it knows I need to use observables, the agent will pull in the observable instructions and come up with a better system in the end.
Next up we have prompts and there's a summary in the end. Uh one of my favorite ones that I worked on is a data prompt. So we do collect telemetry in VS Code and you it's in the open source so you can see what we collect. You can even get a CLI command to get all the telemetry events we send. It's usually around like what features get gets used and and how high level details. But we we also as a team want to democratize access to learn from that data. If you work on a feature, can you make sure that it's fast enough that you have the right entry points, people actually find it. and to not have to marketatize it is actually a great way to put something that's not coding into the agent that you can then formalize as a prompt so everybody can run it without learning custom and all the data tools you would need to access the data. In this case, we have an MCP, the Azure MCP running behind this and access to the GitHub repo to access another separate repo and it just describes the flow of how to write custom based on the data and what the typical flow is we want.
We have a few other things. Um, I wanted to show off the plan mode we're using, but they actually shipped that now. So, let me show you the plan mode that's now built in. Who has seen the planning keynote stuff? So, really exciting. Um, so plan mode is now here. Previously was down here, but you can actually see it. If you ever wonder how it looks like, you can just open up and look at it. Uh, which tools it has access to and make your own. I really encourage you to think about how you want to plan. Is it creating a branch? Is it looking at confluence? Is it uh creating a draft that you want to share on Slack or back into the issue? There's many ways people are figuring out planning and like AI first welp overflows. So it's really important for you to to figure out what this means.
So we have uh the important pieces here uh is there's some UI elements description and argument hint which makes it nice to use which tools it is restricted to. That's a clear context engineering pattern restricting the amount of things the LM can access the agent can do and then handing the handoffs. If you now start a plan down here, um, editable to editable to-do list, you would go into plan mode and in the end, maybe we can do one here. Let's say you see the other context engineering pattern that's being used here is run sub agent. And this is down here. You can actually mention tools in the description and the isolated sub agent in this case will actually do all the discovery. This little UI pattern here is means that this agent is now running the surges in its own agentic loop and just returns the right amount of data to the parent process. So I've been using this in all my prompts. Now when I do research, I just point it towards user sub agent, do a lot of research, go really deep and just bring back what's needed to answer the question. And that way I have extremely long running agentic conversations because they keep reducing the amount of context that gets built up to just the essential elements. So it reads now a lot of files, there's a lot of expiration, but it just returns what's needed to implement this feature request.
So lastly, um, so we talked prompts, we talked instructions. Let's just look at how it looks like in the real world. So custom instructions are the kind of the rules of engagement always on and coding standards. That's the sweet spot. Don't do role play. Don't give it as a TypeScript expert. Uh, just to focus on how to write good code, what is bad code it should avoid, which tools to use, and where generally things are in the codebase. Best case, you just let Copilot write it for you. There's a generate chat instructions button right up here. It's a command. We have a nicely fine-tuned prompt that will rerun for you and we generate instructions in your codebase that you should review, but it's a living document. You keep updating.
Next one is prompts. I mentioned it's these oneshot reusable commands. Think about what things do you want to automate. I have many. I have one thing that just commits um just basically takes all the changes I have makes a commit from it and pushes it. One thing that commits and makes a PR from it from a basic perspective. So all the things I don't want to spend time in CLI all the things I don't want to spend time to research I have started to put into my own personal prompts and that's where I recommend to start as well. And then custom agents are these really rich personas workflows where you want to spend more time iterating in a workflow that become multi-step constraint to tools and their own kind of system prompt and kind of workflow.
Okay, let's go into some more specific workflows very quickly. Um, so one I want to show off is the test driven development flow. I have TDD red which writes failing tests. I have TD green which writes passing tests which makes the test faster by adding implementation and then TD refactor which uh makes the implementation nicer and polish it up because you just want to have the MVP in a green phase. So all of these are connected with handoffs and you see that over here. Just going to make this bigger. So this is a TD prompt which is the entry point. Let me show that one quickly. It just says TD start with TD red start new feature describe the feature we're going to go into TD red. So you can think about prompts and modes as composable primitives that give you different kind of entry points.
If you look at this it started the TDD. It noticed there's no TDD artifact yet where it documented what it's supposed to implement. It creates it uh by looking at the codebase creating the TDD documents. Let's look at it quickly how it looks like. So it's usually just closing stuff. >> It's a list of tests, edge cases, design notes and where's effective needed and then the done list will be slowly build up and that's how these these modes work together. TDD now picks the ne next test and then ask for the user to actually move on to implementation. Now if you look at my tests I have one failing test that it just created in TDD red and that's what I can review now and that's where modes really shine and handoffs that you can have a moment to have human agent cooperation reviewing the test as a contract that it makes sense and then moving on. So think about what you can automate in multi-step flows where the human in the middle is still able to review and accept and go to the next step versus other agents that are more orchestrated and just move on without you. You want to have that control.
Next up we have more visual flows. Um, just want to show off if you think about plan mode it's now built in. It doesn't mean that you're stuck with it. Think about what are the entry points and how you can scope plans out of the box. In this case, I wrote a planned spike problem which is really about focus on the architectural proof of concept. I don't want you to build a whole feature and write tests. I just want to think about how will the back end work. What are the critical points in this plan that I need to reason about more often times that's either the back end or infrastructure or I also have one because I keep having this problems with plans. I cannot imagine what the UI looks like from a big markdown dog. So think about how modes and plans can work together to bias the planning towards what the most critical thing is you need to figure out. And lastly, think about how you can work on UI things like this where it can now move into a designer mode and then start and then have it play around with what is actually implementing and figuring out live what's happening. So breaking down into more interactive steps that you can control. So this can run uh, you can hook me up later with to see how it looks like in the end.
So start on the left side custom instructions uh bootstrap minimal fix the agent automate chores and then move on to workflows. So that's that's how you start uh a lot of documentation in our VS code docs. There's a doc as well about context engineering. If you want to read more it's written by me. If you have feedback uh please open issues as well. Otherwise happy context engineering.