Transcription
Hello. How's it going, Olivia?
>> Good. How are you?
>> Good. I mean, like obviously I know everything that's in the release, right? But when I sit through this call and I was just like backstage, you're just like, "Wow, there's actually a lot in this release." And I was reading through the release notes this morning. It just makes me super excited.
>> I know. No, I feel the exact same there. There's always so much more than you think even. Um, and I mean, we like that was just like three three people's demos, right? That's just like I feel like touching the surface of everything that's in the release. Um, I know >> check out the release notes for sure, um, y'all to see like everything that's in it. Um, but I totally agree. I feel like people, you know, right? We're in the product every day, so we kind of like feel like we are an expert, but there's always just so many things like, "Oh my god, I totally forgot about that." Or, "Oh yeah, that made it into this release." Um, so yeah, we get excited about it too, y'all. It's so fun. I mean, like literally, like I go to standup, I look at all the merge pull requests every day. I try I use VS Code every day, VS Code Insiders, and like I still manage to miss things. So,
>> yeah,
>> we're gonna we're going to share a screen.
>> Um, and then Pierce, I know probably most people know who you are, but if you could just give like a super quick intro in case we have any uh new folks on the call for
>> Yeah, I'm Pierce. I'm the product lead for VS Code. So, I joined the VS Code team last December. I love engaging with all of you on Twitter. So, I'm at Pierce Boen on Twitter. I'm also on Reddit from time to time. So, uh, yeah, if there's ever any feedback you have about the product, of course, you can use our regular channels like GitHub issues. Um, but we also look at we look at all the YouTube stream comments, we look at Twitter, we look at Reddit, uh, we look at Hacker News. So, if there's any feedback you ever have, please don't hesitate to reach out. Also, this is awkward, but I have to restart it. We can just read recitions. So, I'll be right back.
>> Okay, that sounds good.
Okay. Well, in the meantime, um what I actually want to do then because we were just talking about this anyways, um is just go through our release notes again. Um so, let me go ahead and share my screen out real quick. Um just so y'all can see everything um that we're talking about, right? Because as Pier said, there is so much more than you think. Um and there's actually, if y'all haven't kind of been in the release notes in a little bit, um we even kind of changed the uh format a little bit. So we made it a little bit clearer of some key highlights that you can jump to really quickly. Um and so you can see where actually we already covered some of these, right? So letting VS code select the best model with auto model that um uh new just demoed. Um and then this contribute models through VS Code extensions is actually what we're about to talk with um with Pierce and friends as well. Um so yeah, definitely check out these release notes. There's a whole list um that you can go here and then as Pierce said like feel free to reach out to him, reach out to me, reach out to anyone you've seen on the on the call today and we're always happy to kind of have that that conversation with y'all as well. I think Pierce is back with that.
>> I think question mark. I'm sharing my screen to screen to Streamyard now. So, we should be good to go. Maybe introduce What are we showing off?
>> Um no, we I did like a quick little teaser while you were off. Um, so if you want to kind of really give the overview, Pierce, that would be great.
>> Yeah, sure. Um, okay. So, we're just in chat here. Um, actually, like I was joking with Cynthia, who's on our team, like right before this, like I was literally using agent mode to fix the bug in my demo. I was like, how much more time do I have? Uh, so I got something going right here already. Um, but I'm in agent mode. So, we just saw new show auto mode. So, that's in the product. Uh, we recently shipped GPD5, GPD5 mini, uh, Croc code, you know, we have Tim and I25 Pro. These are all the provided models you get with your copot subscription. I think it's pretty cool. You get all that for 10 bucks a month, right? Or more depending on your plan. And also BT Dups, if your model picker doesn't look the same as mine, you can go manage models here, co-pilot, and you can actually like go in and say, "Oh, I never use like for many or whatever, right?" Okay, let me just get it out of my list. Like, and so for me, these these models are like my daily drivers. and increasingly like auto, right, for the reasons that N showed off, but like it's very common that people want to use models that we don't have in the box. So, how do we enable that? Um, so bring your own key. Um, so we've had bring your own key for a while. Um, so if you go to click the models drop down, this looks like a bug. Look at that theming difference. See, this is also why I love the live streams, Olivia. I get a little
>> Yeah.
>> Yeah. It's like it's like magnified. As soon as you like demo to a wider audience, you like immediately see like little bugs and little things that happen.
>> Yeah. I mean, it's kind of funny. We're hiring for like a product manager right now and someone asked me to describe our philosophy and I was like, well, it's kind of like like I used to be an advocate, a dev advocate back in the day when I worked at Sam and it's kind of like advocacy driven product management. Like we take the product, we go show it off. We have our own learnings from building the demos. Like customers give us feedback, developers give us feedback from the VS Code community and we make the product better in that way. Uh but anyways, this list, if you've used this before, uh has gotten a lot longer. So, before we used to have like some built-in BY providers. Uh, bring your When I say BY, I'm just saying bring your own key. Kind of like bring your own beer, Bob, I guess. Yeah, love it. Um, Burke and I are big fans of 0% beers. Um, so, if you know any good ones, please send them to us. But we had some models, some model providers already in here. So we had like Azure, Anthropic, Olama, Open Router, Google for Gemini. So like we had some built in but I think we realized this process doesn't scale right? So a couple problems like one we have to personally go and write a provider for each like different possible provider that exists on planet Earth. That is obviously quite expensive. We would frequently get feedback like oh like why don't you add these people or why don't you change this so that this works and it's like we would love to do that but there's just such a long tale like there's so much happening in the space that like the amount of providers we would have to support is like insane right
>> so that doesn't scale and the second reason is like while we read docs and we try to understand like the different providers that we integrate with like it's hard for us to say we're going to write as good as an open router provider as the open router team who understands all the nuances of their APIs what exactly to show off we wanted to basically say how can we enable the community to contribute models to GitHub copilot um and so we've introduced this new bring your own key API and so essentially you can build a VS code extension and uh you can ship that VS code extension and basically once you do that it can contribute a language model uh to GitHub copilot so you can see I have a couple on my list here and they'll show up here and then so you can see I have like Cerebrris hugging face AI toolkit those are new as of today and so those are extension contributed bring your own key providers so those aren't actually built into VS Code or GitHub Copilot chat, but they're actually coming from an extension you've installed, which is pretty cool.
>> Cool. So, you literally just can go install the extension as normal and it'll show up in the right.
>> Cool.
>> Yeah. So, look, the hugging face people have been updating. Look at that beautiful
>> Oh my gosh.
>> But like I just installed that right before. So, they're doing like lots of live updates, but like
>> Well, so I think that that's a really cool thing to call out too, right? It's like not just is it on the VS code team when we were managing all the different providers like okay we need to figure out are we going to do it well but it would take us longer as well too right whereas the hugging face team is just they're able to just do us right now we get those updates right away you don't
>> even if we had the time like they usually can ship whenever if there's a bug it's on the release train whenever you ship an extension whereas VS code like while we ship insiders every day we don't ship stable that often so if there's a problem in stable like sure we'll if it's a major issue we'll get into a recovery release but not always so yeah it's super cool like um here's this rebris extens ion. So like one bit on discoverability you might be like pierce that's cool but like how the heck would I know to go install the cerebrus extension because it's not immediately obvious from this manage models experience. So I can come in and I can basically go I want to show like the ideal experience we're working towards. Okay. So, you know, if you go to the color picker, like we have like browse additional color themes up here and you can kind of search and this is actually fetching real themes from the BS marketplace, right? And we also have some built-in ones here and then we populate with some like other ones that are kind of popular from the community. So, we want something similar here where I could type like for by I could type like cerebrus and then even though it's not like showing by default in this list right here, it will actually get populated in. So the idea is like even for extensions contributing models, we want it to feel like it's built in and then we'll do the work to make the discoverability of these things better over time.
>> That makes a ton of sense. Yeah, I love that that flow. Do you have any sort of like insight on to like when that sort of flow might land?
>> Uh well hopefully this next month we have actually an upcoming VS Code team week. The whole VS Code team is getting together. We're kind of all over the place between Europe and the US and so we're all getting together for kind of our annual thing next week actually. So, this month is a little bit of a weird month in terms of commitments for delivery because we have a lot of people traveling and we're doing we obviously when we're together want to spend time together. So, um yeah, we'll see. But the plan the plan should be up on Monday and I'm hoping we can do this. Um there's
>> in the near future though at least for like a six months out thing.
No, no, it's not a And if it's going to take that long, I'll by fit something and submit VR. But also, like we could do like a little filter thing here too, like in the marketplace where like category, maybe we already have it, now we don't. Like a language model or something. So that
>> tag for language models.
>> Exactly. So I thought what better way to show off the bring your own key stuff than a personal extension I built using this new bring your own key API.
>> I love it.
>> I'll actually explain how I did this. I took the we have a an extension samples repo for the VS code team. It's public and there's a chat provider sample. It kind of has the basic, you know, building blocks of what you want with like a fake provider. So I basically took that and then I wrote a prompt file and I basically said, okay, how would I transform this sample to work for any provider, right? Step one, like I need to give the provider the models. Okay, so what are the docs that this thing, you know, this provider has? How do I actually integrate with it? Is it OpenAI compatible? Right? um does it need an API key? Okay, if it needs an API key, we need to put it in secure storage, right? And so I just basically defined a prompt file. And then it was really cool because I've actually been like exploring building a lot of these providers because it's actually pretty formula. And so this is like a cool use case for the prompt file stuff where it's a repeatable scenario that I'm like kind of running through again and again. And then of course like it's not perfect. I have to do some tweaks for each provider, but it gets like the initial implementation done, which is pretty neat. So basically, there's like a couple different things. Okay, we need to fetch the models that show up in the list. Makes sense. So we have some basic like you know HTTPS requests here to actually fetch the models pass tokens. Then we have to map kind of the fireworks data structure model to like ours and that basically gives all the actual models that you show in the drop down which I'll show in a second. And then there's like other stuff you have to wire up like basically like how does the actual let's see yeah, provide language model chat response. So this is like actually the the part where we form a request to fireworks is the provider that I built in this case and we actually okay handle the chat request because when you use bring your own key Olivia it's actually talking directly to that provider that you've configured from client right um there are some situations where it goes to the GitHub API like for example intent detection like if you're in ask mode like if we you know we how we have the participants like Azure string or whatever
>> so that that thing will still hit the cloud endpoint so we don't say like 100% like It's only talking to your thing, but like for the meat and potatoes of what's actually happening in chat, it's using your thing. So, you have to actually go pipe that logic in, right? That's what I'm doing here, or I guess agent mode did for me here.
>> Yeah. Uh, let's actually run this thing.
>> So, so kind of on that note too, like because I I know um at times like one of the challenges too is like if you're just talking to the provider, then like having VS Code kind of ingest like the response correctly in like a user friendly way because some things can turn different responses and like error codes and things like that. Is that all part of this API as well too? So you can kind of make sure you're ingesting giving like friendly messages if needed when you're talking.
>> Exactly. Yeah. So there's like any weirdness that you want to handle with your provider, you can do that. I'll give a really good example like pretty much every provider says we're OpenAI compatible as like the way we talk to LM back and forth, right? In practice like that may be largely true, but every single provider we found has like little nuances of like oh like this is an empty array when it should have been null or something like that, right? And those are little things that like were always gotchas first when we're building these providers. So it's not quite as simple as oh like it's the same OpenAI compatible like logic, right, for each thing. So each provider has like little nuances and how they
>> Speaking of new features, Olivia, I'm going to detour. This is a classic Pierce detour. Look at the chat. Do we show this off in the stream yet? Look at this. I'm in chat. I can see a history of my past messages. So I think this is actually a new setting. Empty state history. So, show recent chat history on empty state.
>> I had not known this yet. This is great.
>> Okay, cool. Yeah, I love doing these demos because you just like are like, "Oh, I should probably show that off." Like, you could always get to this via this history button that was like up here, but like often like we had all this like white space just from a UX kind of consideration standpoint or like, well, like a lot of times I do want to like return back to a previous place I was, right? And so, it just kind of makes sense that we expose that there. And also, a lot of people told us, why doesn't chat support history? And we're like, we do.
>> Yeah. Oh, okay.
>> Um, anyways, um, okay. So, I built this extension. I'm going to go look at look at all these bring your own key things. So, anything below this, like this indicator right here is all bring your own key stuff. So, I have stuff from different providers. Uh, so I have Deepseek, I have GPD OSS like a local model. I'll talk about the custom OAI thing in a second. I have GR codefast one, not the one that we provide in Copilot, but directly hitting the XAI APIs. I have this from firework or sorry from hugging face and then uh I'll show off how I got this one. So I come in here. I've just built this fireworks AI provider. And so this is kind of the logic that I wrote for my extension, right? Like okay, I say what is the name? Okay, fireworks. Okay, that shows in the first list. Then I actually hit the fireworks API to populate all the models. And you can see here like this is where it's like it's useful to know the provider and be the provider to write this because I have like a lot of other random uh things in my provider list here. So I'll just kind of come in. I have uh this model selected in DeepSeek. I don't know how I say this. Like it's a very popular model in OSS like Kimmy Kimmy something in my list.
>> Say Nim. But yeah, now that you say that, I don't know if that's actually how you're supposed to say it.
>> So then I'm going to come in and I'm just going to like I select that model and I'll just say like let's just do a simple test. Hello. And so what this is actually doing is this is going in. It's going to be a little slow because I'm hitting I'm debugging an extension, right? and actually hitting this. But this is actually yeah, this is going to form a request to chem K2 instruct on fireworks, hosted on fireworks, um, and actually return a response back to me. And of course, like if the model supports tool calling, it would also show an ask mode in this or sorry, an agent mode. In this case, it does. And so I could just go in and also use it in agent mode if I want. So
>> very cool.
>> Yeah, this is super easy API to work with. And so if you're if there's like you know an inference provider that you're like I would really love to use that inside of VS Code like go build an extension ship and I love this because it's like the classic ethos of VS Code right like choose your own adventure extensions like make it your own right like that's why I love the customization features we have for things like chat with chat modes prompt files stuff like that and now like with the bring your own key stuff it's also like choose your own adventure right like whatever exactly setup you want VS Code enables you to get that.
>> Yeah developer choice all about it. Uh, and just to to iterate because I don't know if we like explicitly said that. So, we kind of refer to this like AK's like BY, right? So, BY API. Just so if you're looking through our docs, it is officially called the language model chat provider API. So, if you're like, why can I not find a good B API? It's called the language model chat provider API. So,
>> branding
>> call that yeah to call that out. If anyone's in the docs and like wait, what is this actually? So, BY is how we're going to be referring to it. And I'm always like I'm very conscious of the fact that like we use a lot of acronyms at Microsoft and I try not to use them. Although in this case I think it's somewhat self-explanatory.
>> Okay. One final thing and then we're gonna bring on a guest. Um I mentioned this custom OpenAI compatible endpoint thing. So I mentioned I I mentioned that um many of these providers actually speak OpenAI compatible syntax in terms of their API for for talking to their models. And so we have this new OpenAI compatible option here. And so you can basically go in and if a provider is not supported if you don't want to go build an extension which you should your break glass in case of emergency is this open AAI compatible option. Now it's using like I said every provider is a little different in how they write this implementation. So it may not work amazing for your provider and so that's the kind of the benefit of the extension path but if you're like I really need to test this and I don't have support for it you can do that. So if I actually go to my settings and I go to open AI compatible models, you can see basically here that I can come in and I can specify other models. So in this case I'm saying like another one I did I did fireworks directly to the API. So you can see I have to supply the earl here and then I just supply some information about the model. So like what's the name? What's the input output tokens? Does it support tool calling? So if that was false it wouldn't show an agent mode, right? Does it require an API key? Does it support vision? Right? And so you just go in and kind of fill this out. It does support API keys. So if your provider requires API keys and the first time you use this model, we'll actually prompt you to give us the API key, we'll put it in secure storage locally, right? Then you can use it. So this is like your break class in case of emergency. The one nuance with this is this is only available in VS Code Insiders right now. This didn't go to stable with us today. It was in insiders last month. We found some issues in our testing during in-game. Just want to make sure it's a really good experience before we push it out to stable. So there is like a little nuance of like you might have some some issues with it. And if you do, please file a bug so we can make sure it's really really good and we get into the next table. So yeah, that's bring your own key.
>> That sounds awesome. Um before we move into kind of our partner here, there's one question I think you're very well suited to um answer just while you're here, Pierce. But Matias asks, you know, what you know, you mentioned experimental settings. A lot of times we see settings or experimental. What's the difference between using VS Code insiders, which you just mentioned experimental settings and the regular settings? If you could just clear that or quiz.
Something we're trying to do better about on the VS Code team is like our settings like expansion. Like there are so many settings. It's hard to keep track of them. It's hard to understand like what state they're in, what that state means. The defaults often like it's like I can't tell you how many times Libya I get in a call with somebody or I'm on Twitter and someone's like why doesn't VS Code support this? And it's like it feels like a face palm to me, right? Because like we have it and we're not doing a good job exposing it behind some setting, right? And so experimental means we've just added this thing and it could change. So actually a really good example is we added specific auto approvals for terminal. So you can apply a regatic expression and basically say like these are safe don't allow these right. So we actually iterated on that setting over the course of a month or two and things changed, right? And it broke. And so experimental basically means this setting, we're still figuring this out, right? And it could break. And experimental settings can go to stable. If we want kind of a broader population of feedback on that setting or we think it's sufficient that it can be advanced to stable, we'll keep it experimental and move it to stable and maybe it's off by default, right? Um then we have preview, right? Which is kind of the next stage of like, okay, this is like something we feel really good about. we're probably going to finalize soon as a setting or just build it in as a default behavior. And then finally, right, like it's a it's a GA thing. And we kind of think about preview and GA features. So specific with Copilot, if you're in a big team, there is a policy that your admin probably has called editor preview features. We basically have to declare these are the preview features we have. So like auto is a good example. That's a preview feature. And so if your enterprise has disabled preview features, some of these features you won't show up. So there's also some accounting we have to do so that like people who want to kind of try all these features in their enterprise before they actually roll them out can do so without it just being immediately available to everyone. That was a long-winded answer, but that's kind of like how we're thinking about settings right now.
>> No, I think that's super helpful. I'm glad that you're able to answer that. Um, and that was the perfect time to ask that question, Matai. So, thank you for doing that.