Transcription
Get ready to supercharge your coding experience. Microsoft has just launched a groundbreaking AI toolkit for Visual Studio Code that lets you run local and cloud models with ease. Imagine creating custom AI agents, running bulk tests, and fine-tuning models all within your editor. And yes, you can access top-notch models like GPT5 for free. Stay tuned as we dive into all the incredible features and how they can transform the way you code. Let's get started.
Hello everyone, welcome back to the channel. In this video, we are going to talk about the fantastic Microsoft AI toolkit. This toolkit is an exciting extension that lets you use both local and cloud models right inside Visual Studio Code. It really simplifies the process of giving custom instructions, downloading, and installing models locally through a user-friendly graphical interface all within your VS Code. You might find it similar to LM Studio, but it comes with some brilliant upgrades.
Now, if you haven't heard of Microsoft's AI toolkit before, let me explain. It's primarily a Visual Studio Code extension that allows you to run models locally using tools like Olama or ANX. You can interact with those models and it even gives you access to almost any model via their GitHub models option. This includes amazing models like GPT5, Claude, and many more, all for free.
While many features were already available, the new addition of an agent builder is really something special. This allows you to create your own custom AI agents directly within a graphical interface, and you can give them access to custom tools. You can write your own code for those tools or integrate an MCP server, making it super flexible and powerful. With this capability, your agents can perform specific tasks such as being a simple web scraper or even a code interpreter.
And let's not forget about the new bulk run option. This nifty feature allows you to run batch prompt testing across multiple models simultaneously. If you're anything like me and love testing models, this means you can input all your prompts and easily compare model performances. How interesting is that?
Moreover, we have the model evaluation option. This feature enables you to run multiple prompts, similar to the bulk run, but here you can create data sets with questions and expected answers. The toolkit will check the similarity in the model responses, score them, and allow you to customize the setup. Extremely handy.
Fine-tuning is now available right from within VS Code. And you can do it in just a few clicks, which is incredibly useful. And there's a new tracing option that collects and visualizes trace data, giving you deep insights into model behavior and performance.
Now, let's get into how it all works. First of all, you need to install the AI toolkit from the VS Code extension marketplace. And if you already have it, please make sure to upgrade to the latest version. Once you've got that set up, you'll see the toolkit option in your VS Code sidebar. Navigating is a breeze. On the left, there's a sidebar where everything is organized. And on the right, you have your main pages.
Let's start with the model section. Here, adding models is easy. You can use various runtimes like ENX or Azure Foundry. Plus, you can add any OpenAI compatible API model right from this interface. Isn't that fantastic?
Once you've added your models, head over to the playground section. It's similar to what you've seen before, but now if you log in with your GitHub account, you can select models directly and hit browse models to discover even more options. You can even access GPT 4.0 and GPT5 for free. How brilliant is that? There are interface parameters you can set, and you can attach images, documents, and even code to your prompts. You can also make the models perform web searches. If you generate code, there are handy add and copy buttons that make things so much easier.
Next, let's move to the agent builder, which has quickly become one of my favorite features. The agent builder lets you create AI agents tailored for specific tasks. You select a model from your available options, input the system instructions for what you want your agent to do, and you can even include dynamic variables using double curly braces. This creates a variable prompt that you'll fill in each time you use the agent. You also have the tool option where you can add tools that your agent can use. You can choose to integrate an MCP server or define custom tools, which is absolutely perfect for advanced users. Once you've set everything up, you can interact with your newly created agent however you wish. The toolkit even features example agents like a web scraper to help you get started.
Personally, I've enjoyed using it with tools like deep wiki and web search, which act like a context engine, providing focused documentation and information. And there's a built-in playwright web scraper that works wonderfully for retrieving documentation and condensing it.
Another great feature is the bulk run tool, where you can create a comprehensive list of prompts and execute them all at once. This is just so useful. The evaluation option is intriguing as well. You can craft your own evaluations for AI models by selecting the types of questions you wish to use. All it takes is entering a question, the correct answer, and then reviewing the model's response. The system will calculate accuracy for you, which is handy.
Finally, we have the tracing feature, where you can view logs of your AI applications and agents. This is fantastic for debugging and seeing the inner workings behind the scenes.
In summary, this toolkit is simply incredible if you enjoy working with local models, testing multiple models, or having a chat interface right in your code editor. It does not utilize system prompts that can slow down local models like other setups. Instead, it runs the models directly, resulting in a much faster experience. Plus, being able to fine-tune models from the same interface is a substantial advantage. So, if you're keen on exploring local models, running batch tests, evaluating performance, or building custom agents, this is a superb tool to have right inside VS Code. It's powerful and absolutely worth checking out.
Thanks for watching, and I hope you found this video helpful. Don't forget to subscribe for more exciting content on AI and technology. See you next time.