Transcription
[Music]
[Applause]
Hi, welcome to another video. So, if you remember about 10 months back, I had covered Microsoft's AI toolkit that allowed you to use local and cloud models within VS Code where you could give it some custom instructions and you could also download and install models locally quite easily with a simple GUI inside your VS code. It was something like LM Studio, but since I covered it last time, it has gotten some awesome new upgrades and it really is a good option to have in VS Code if you like to use local models or things like that.
So if you don't know about Microsoft's AI toolkit, then it is basically a VS code extension that allows you to run models locally quite easily via Olama or ENX and things like that. You can then chat with those models and it also allows you to use almost any model via their GitHub models option which actually gives you access to models like GPT5, Claude, and more. for free.
These things were doable before as well, but now it has something new, which is an agent builder. This allows you to build out your own custom AI agents inside the graphical interface by giving them access to custom tools. For these tools, you can write code yourself, or you can also now integrate an MCP server and give it access to MCP tools. This allows it to do certain tasks that you want. For example, you can build a simple web scraper tool, a code interpreter tool, and things like that, which is definitely pretty cool. It's really useful, and I'll get into it.
There's also a new bulk run option that allows you to execute batch prompt testing across multiple models simultaneously. So, if you're a model tester like me, then you can dump in all your prompts and compare model performance across different models, which is pretty interesting. There's also the model evaluation option, which allows you to run multiple prompts just like bulk run, but here you can build data sets with things like questions and expected answers. It will check for similarity in responses, give them scores, and you can change that setup as well. You also get fine-tuning options, and you can now fine-tune models directly from VS Code with just a few clicks, which is pretty interesting. Then there's the tracing option, which allows you to collect and visualize trace data to gain insights into model behavior and performance.
So, these are the things on the surface. But now, let's get into it and let me show you how it all works and what's exactly new here. But before proceeding, let me tell you about Ninja Chat. Ninja Chat is an all-in-one AI platform where for just $11 per month, you get access to top AI models like GPT40, Claude 4 Sonnet, and Gemini 2.5 Pro, all in one place. I've been using Gemini for quick research, but what's really cool is their AI playground where you can compare responses from different models side by side. Their mind map generator is a gamecher for organizing complex ideas as well. The basic plan gives you 1,000 messages, 30 images, and five videos monthly with higher tiers available if you need more. Use my code king 25 for 25% off any plan or king 40 yearly for 40% off annual subscriptions. Check the link in description to try it yourself.
Now, back to the video. First of all, you'll have to make sure that you install the AI toolkit from the VS Code extension marketplace. And if you already have it, then make sure you upgrade it to the latest version. Now, you'll see the toolkit option in your VS Code sidebar. And from here, we can easily go ahead and use it. This is what it looks like. On the left you have the sidebar for everything while on the right you have the pages.
So first you get the models option where you can easily go ahead and add models via multiple run times like ens or azour foundry. You can also add any other open compatible API model through here as well which is awesome. You can use these models in the playground section available here. It's pretty similar to the last time. And if you've signed in with a GitHub account, then now you can also choose models directly here, as well as hit the browse models option to check out even more models from different providers. You can use GPT4.1 and GPT5 for free here as well, which is pretty cool. There's also the inference parameters that you can set and you can attach images, documents, and even code. You can even make it perform web searches for you, which is also very cool. If you make it generate code, then you also get add and copy buttons. The add button creates a new file for you to save it quite easily, which is really helpful if you'd like to generate code quickly and use it right away. You can also hit this get code option which will give you a simple open AIS SDK script to use in your projects.
If we move on from the playground then we also get the agent builder which is a new feature and this is one that I really like. What this allows you to do is build agents for specific tasks. So you select a model here from the available models. Then you can put in the system instruction of what you want the AI agent to do. You can also put in dynamic variables in the instructions with two curly braces and then it will create a variable prompt for which you'll need to enter a value each time you use it. Then you've got the tool option. This allows you to add tools for the AI agent to use. You can either add an MCP server for it to use as a tool or you can define custom tools, which is great for more advanced and custom usage. If you want to use an MCP server, then you'll need to add it from the left sidebar where you can see the MCP server option. Here you can add any kind of MCP server setup and it will show up for use. Once that is done, you can go ahead and chat with the agent that you've created and use it however you want. They've also included examples for things like a web scraper and more. I've personally been using it with deep wiki and web search tools, treating it like a context engine that helps me get focused chunks of documentation and information. You can even use the Playright web scraper here, which is really good for fetching documentation and compressing it.
You also have the bulkrun option, which allows you to create a whole table of prompts and then run them all at once, which is super useful and definitely cool. The evaluation option is also pretty interesting. It allows you to build your own evaluations for AI models by selecting the types of questions you want to use with it. For example, you can use it to measure tool calling accuracy and much more. You enter the question, the ground truth, and then check the response. From there, you'll see the accuracy calculated for you, which is definitely great. You also get the tracing option to see the logs of your AI apps and agents, which is also very useful. It's really good if you'd like to use local models or test multiple models. And this is even better if you want to have a chat interface right from your code editor without the system prompts that things like rue use, which often make local models super slow. This setup just runs the plain models with no system prompt and it also allows you to do fine-tuning through this conversion option as well. This is really a great tool to have in your VS Code if you like to experiment with local models. Overall, it's pretty cool.
Anyway, share your thoughts below and subscribe to the channel. You can also donate via Superthanks option or join the channel as well and get some perks. I'll see you in the next video. Bye.
[Music]