How To Build a Hybrid AI System with Any-LLM (ft Nathan Brake) - Ep 81

There are so many LLMs to choose from these days. Do you know which model is best for your use case? Can you easily switch between models to try them all out? Any LLM is an open-source offering for Mosilla AI and it solves this problem in an open way. On episode 81 of Tool Use, we're joined by Nathan Break. He's a machine learning engineer at Mosilla AI and the creator of NLM. And today we're going to discuss how to implement Nelm, why you should try out different models for all of your AI tasks, and how you can save money while being more secure. So please enjoy this conversation with Nathan Break.

Misilla AI, the first thing everybody thinks is like Firefox. What what AI features is is Nathan putting into Firefox? And the answer is nothing directly. Uh, Misilla AI is a startup that was funded by Misilla, the main organization. Um, but we don't work on Firefox in particular. We're kind of working to try to help support the AI ecosystem of open source specifically. And I think, you know, in 2026, in the end of 2025, there's been a a big push worldwide for open source AI and how important that is for each country uh in the world and just people feeling like they have control over over the AI that they're using. And Misilla AI's mission is really to invest in that open source ecosystem and to help be a leader in, you know, promoting choice and transparency in the way that we're using these AI tools every day.

Love it. It's such a good mission to be a part of. And would you mind telling me about NLM?

Yeah, absolutely. So, I I wanted to talk a little bit about NLEM today, especially in the world of choice and open source. Uh, NLM is a project that we we named with not much nuance in that the idea is that you can talk to any LLM and boom, product name. This is why engineers shouldn't always be in charge of product names because we're not the most creative with naming stuff. But the the idea of any is to help support people that are building applications to use a single interface to talk to any provider. So especially the reason a lot of people go with the proprietary models like OpenAI or Anthropic is because it's so easy. Like if you want to talk to GPT5, there's basically just two options. You either talk directly to OpenAI or you can you know work through Azure to uh spin up GPT instances. If you want to work with a leading open weight model like a Mistral model, there are basically infinite options about how to connect that because you have to choose where you want to host the model. You know, do you want to talk directly to Mistral's hosted service? Do you want to run it on a different third-party cloud hosting service? Do you want to run it locally? Do you want to run it self-hosted? And it gets really complicated and that's part of the the good things about choice in the LLM open weight world. Uh, but the downside of just like it can be overwhelming to feel like you have too many options. And so that's what any LLM is trying to help bridge the gap between to help make it so that you can design a system that points to an LLM that's hosted in one place and a model that's in one place but not lock you in code-wise to always have to use that model. So any LLM was kind of born out of my own smashing of the keyboard and frustration when things didn't work. Uh, when we built an integration for OpenAI and then we were like, hm, we should also be able to talk to Mistral and then we built the integration for Mistral, but there was one little tweak where you had to deal with this parameter differently. And then you started adding different providers and everybody just had a little bit of a difference of how to talk to their providers. And then you look around and you're like, wait a minute, everybody is also adding these patches to their code and seeing the value for a central uh place to put that code and and a library, basically. And there are other solutions out there that are doing uh trying to help support this single interface to talk to different large language models. Um, but we wanted to focus on a specific ecosystem where our library NLM is as small as possible. That my dream would be if one day NALM the SDK goes away and it doesn't need to exist because all the specifications have been just so solidified and the world agrees, we have world peace, everyone's happy and agrees. Um, I don't think that'll happen. But the idea is that we want to make this as simple as possible and outsource as much logic as possible to each of the actual providers. So NELM is a Python SDK, a library and it relies upon each individual provider's published SDKs that they publish themselves and using that. So open, OpenAI will publish their own Python SDK. Mistral will publish their own Python SDK and each of those have typing and uh a lot of definitions that they'll tell you here's how my code works that you can then tie into to make sure that things work correctly and get really quick signals when that provider updates something that breaks your code. So the idea of ALM is just to make it as simple as possible, be as little code as possible and rely on the work that each provider is already doing on their side. And then if we find an issue, we can report it to their code and they can fix that on their side and help everybody, not just any LLM and you know, just improve the ecosystem in general.

It definitely makes it easy. Uh, one thing I found is a lot of people, they're just content using a single model. So could you go into some of the benefits of the ability to rapidly switch between models and explore from the different providers?

Yeah, absolutely. I mean, I think just in the past six months, there are a lot of people that you'll you'll have a discussion and myself included, I will say, oh, but the language models can't do that, AI can't do that. And you think like, I understood in January 2025 the capabilities of the leading large language model and every month all these new models are coming out and now the capabilities are improving and you need to constantly be checking, is the newest model actually working better? And I think even more interesting from the open source AI and the open weight AI world is, are there smaller models that actually do the job just as well as the bigger models from a year ago? And that is helping to power being able to run AI on smaller hardware so that you can start to consider reasonable self-hosting options to run it on your own hardware and not have to pay for it to be hosted somewhere else. And the capabilities, as the leading models get smarter, the smaller models are also getting smarter. So if you only need a certain level of intelligence to do a task, you may actually be able to use a smaller, faster, cheaper model to do that same thing that you in last year needed a super big model to do. Um, so I think it's not only choice but like like the ability to quickly experiment and even across providers, if you, I think it's artificial analysis posts diagrams of the same open weight model deployed in different cloud providers and shows how their performance differs. And it's like, if you're talking to a Kimmy K2 model hosted on Bedrock versus Azure versus somewhere else, they all have different performance even though they're the same model. And it's the same with running local versus running in the cloud. Even on the same model, the performance can be different based on the optimizations you make to save money or or you know, improve speed and stuff. So even in the same model class, you want to be able to quickly switch between um and have that option to switch quickly.

Absolutely. And and just wanted to share a personal thing I built. So a lot of the small tools, scripts, etc. that I have all run through a central AI service that I wrote because I want to be able to do things like consistent logging, consistent retry logic, and just be able to encapsulate that and have everything benefit from it.

But one thing it doesn't do is allow me to just like quickly change the model to test these things because like you said, these small models becoming more and more capable. Even if you don't necessarily have your computer, if you can just point it somewhere else really quickly, you can run tests really fast and say, can I still get the output of this tool that I've been using daily for the last year if I pointed to this very cheap model? And all of a sudden you're dropping cost, improving speed. So, have you seen some unique use cases for uh any LLM that has just kind of either impressed you or just kind of demonstrated that this technology really is enabling new workflows or or new abilities?

Yeah, absolutely. I mean, something interesting about writing an open source library is that you mostly just hear about the bugs because the goal is like, okay, we see people are downloading it and no one has reported any issues and that's a good day. And so a lot of times you don't hear about the success stories as much as as the like, this one corner case doesn't work for me perfectly. Um, but I think we have seen uh just the benefit of people being able to switch fast and integrate it easily into their own applications. So like we just posted a a blog post uh yesterday or a couple days ago about some new integrations we offered with Lang Sheen and uh Jupiter AI, I believe. Uh, and basically it's just, you know, easy integrations so you can use NALM in products that are already there. And I think that's kind of nice of like this, if you're using an application that's using AI, you may just have more choice soon by those people integrating with NLM or other libraries.

Absolutely. And I know NLM is not the first one to do u model routing in rapid model switching. Could you kind of go into some of the decisions that led you to your implementation of it and why this is a good use case for for the majority of builders?

Yeah, absolutely. I mean, there's a bunch of other really good libraries that are working on similar goals and I think that uh our approach was kind of a combination of a few different approaches that we'd seen across the ecosystem. Uh, the the core of the design principle was really to rely on these provider SDKs. Uh, we just saw that like the world of AI is moving so fast and each company is racing at breakneck speed to add new features and implement things that trying to monitor what they're doing and make updates on our side based on what we saw was just it was too much. Like we basically realized that any LLM and some of our other projects were basically just testing harnesses for everybody's provider SDKs because as soon as they broke something, we had integration tests that were running every commit and we'd be like, "Wait a minute, on this SDK that was a patch version, it actually had a breaking change that we saw because, you know, a type thing changed." And that was part of the design principle that we wanted to go with with NLM is just like things are moving too fast. So we need a way to tell us immediately when something is wrong without having to monitor it. Like we just need a flag in our uh GitHub actions, our automatic testing to tell us like this update is now breaking something that you rely on. And I think that has been kind of a differentiator as well. Uh, that's that's like the big, I think focus of of what we're building.

Going to leave this one up to you. Would you rather show us an implementation or talk about NLM gateway?

I'd love to show you an implementation. We have uh kind of three pieces of any LLM to uh yeah, the three pieces that come together to form the whole product currently. One is this the Python SDK and we now have a SDK in Go as well and I think we have some other ones that are are coming soon. Um, and that is just a Python library that you can use in your Python code or your Go code. Then we have the NLM gateway, which is sort of just a proxy server that sits in between the user and the provider so that you, if you're in an enterprise or or a company that wants to be able to track budgets and uh restrict, you know, certain model usage, you can basically manage user keys inside of this gateway and see what tools people are using and uh how much of it and be able to just make sure that one person isn't spending your entire higher budget. Um, and then the third thing that we're working on now, which I'm really excited about, is called the NELLM platform, which I think a new name is pending, but uh, it's in beta right now and it's free to use. And the idea of any LLM platform, which I'll I'll demo in uh a little bit later, is basically it's a platform that we host for you, but we never actually see any of the content of what you send to a provider, but we have a platform that will gather your token usage and be a vault for your API keys so that you can easily track AP or API usage and and tokens and API keys without needing to actually send the data through it. So it's basically just like thing that sits on a side that helps you monitor what's happening without actually looking at the content that's being sent. And even though I think we're looking at being able to add features that let you, you know, host models on the platform, it's really exciting to think about a place where you can easily see the the count of tokens and how much you're spending without actually having to route traffic through the platform. Uh, which enables some really cool stuff which I'll I'll demo in a bit. So yeah, let me just, you know, I love to show code. But I'm an I'm an engineer at heart. So I want to just like show it working and uh I'll try not to get too excited about everything that that's happening because I am definitely I get excited. But okay, you should be able to see my screen and uh I will just I took a screenshot of the splash page of NLM just to kind of give you the the high-level overview of what it's going to look like before I write the code. So the the API when you're writing Python is pretty easy. You basically just import uh the NAL on library. For this demo, I'll show you the a completion call, which is just basically a chat message. You ask a question, you get an answer back. And then I'll show how easy it is to switch to a different provider in a model without really changing your code. And uh let's just get right into it. So what I'm going to do here is I'm going to show how easily we can switch a provider while using the same model. So for this one, I'll specifically use the Mistral's Ministry 3 billion parameter latest model. And this ties a little bit into what we were talking about earlier about the model uh capabilities have expanded so much that 3 billion parameters in 2026 is basically nothing parameter count-wise. The leading models from uh OpenAI and Anthropic are probably in like the trillion parameter counts. Um, so many, you know, uh, many multiples more parameters and I think my expectations for the capabilities of models this small are kind of low where I still think like if I asked it to name the capital of France, would it get it right? And like that's kind of my expectation is like bare minimum basic knowledge. So something to demonstrate the new capabilities is image recognition. So this small three billion parameter model is able to recognize images. And if I feed this image, which is of the Pittsburgh Penguins, uh, because I'm from Pittsburgh, and if I was to feed this image into this language model, what do we expect the language model would actually understand about this? Uh, and I think if I hadn't run this demo before to make sure it worked, my expectations, I don't know, uh, Mike, what do you like, what do you think that a model would be able to recognize about this image?

Maybe like general characteristics, you know, Stanley Cup number 87. Also, bold move showing Crosby after those Olympics, but uh, silver respected angle.

There's a lot here, right? There's numbers, there's, uh, words, there's objects, there's scenes. Like, there's a lot to figure out here. And so what we're going to do is we're going to pass this image into the completion API and we're going to use the mistrial cloud hosted model which I've already set an API key behind the scenes so that I can talk to it and we're going to say we're going to move it to the cloud. We're going to ship this image over by doing something called B 64 encoding it. And we're going to send this image over and we're going to ask, what do you see about it? Just, you know, tell me really just what it does or what what's in the image and then write me a poem about it. And uh I already installed the image or sorry the the NLEM library before, but you would basically just do a pip install any LLM SDK and then I will run this script with uh UV is just a package manager for Python. And so this has got, you know, it's it understands it's a hockey player. It did number recognition correctly. It did name recognition. It did object recognition. Um, which is, I mean, and then it wrote a nice poem about it. And on its own, like this is cool. Like that was pretty fast. This is a model and it, you know, recognized a lot of things that if you're building home products at home, there's a lot you can do with this. Um, but this is still kind of like, okay, cool. But you shipped an image off to the cloud and it worked. Yippee. I can do that with any of these models. Like maybe this is cheaper, faster, whatever. It still cost me something. I had to set it up. But because it's a small model, we can actually run it locally. So, uh, Misilla AI has another project which we started working on called Llamaile. And, uh, if it's in our GitHub repository, it's open source. And what llama file is is basically a project where you can build executable files that bundle in a large language model and can be run on any m machine. So you build a single binary and then you can pass that binary wherever you want, Linux, Windows, Mac, and it'll just run. And I downloaded and uh my uh colleague David helped me create a llama file for this same model. And so now we're going to run this model on my Mac and hopefully my Mac does not explode from streaming at the same time as uh trying to run a large language model. Um, but with just one command, I am now running this Mistral model that we just talked to remotely on my laptop. And now we'll get to talk to this model instead of Mistral. And in order to do that, all we have to do is change the provider to llama file. So I changed one line and I said, "Okay, don't don't talk to mist anymore. Let's just let's stay local. Let's talk to llama file on my computer." And then we'll run that same command to run that client. And now you can see my terminal window making some noise over here because he sent it. And here we go. It's slightly different because uh you do some things to make it work locally which I won't get into, but it's basically the same model and we still got hockey player 87 Crosby and large trophy over the head and I got the poem.

On the note of of llama file, I actually have a UFV key that I've loaded with like hundreds of gigabytes of llama files because I thought, you know, being able to archive this and storage is great. So l is such a cool process to be able to just like pass models around as as binaries. It it was great.

Yeah, that's that's the beauty of it. And I think that's why we're still it it had a bit of a uh a period where we weren't able to maintain as much and we're trying to help, you know, give it a facelift and bring it back up to speed. And I think that it's still a really powerful concept to make language models accessible to people. And that's really the that's the whole idea, just like let's make it as easy as possible to run these things because there's a lot that you can do wrong and let's just like try to make it as easy as possible and that's what Llama File does I think really well.

Excellent. So yeah, so so llama file, cloud models. What other local options uh do people have to leverage um any LM?

Great question. There's also uh the popular ones are LM Studio. Is another like an application where you basically download, you download an application that will download the models and manage them for you, as well as Olama. I think Olama is probably the easiest to get started in if you want to download an application once and then be downloading multiple models. They make it really easy. Um, and LM Studio, if you're on a Mac, has a really great connection to Mac specific uh, model weights that make it kind of optimized for usage on a Mac called MLX. And there's a bunch of stuff that in the Apple ecosystem that makes language models work really well if you use this MLX framework. I think those are the two big ones that I think about. And I'm an LM Studio user myself as well to be able to quickly download and and try out new stuff.

Nice. Um, I've actually so I was historically always using Olama. I've recently switched cuz I'm on a Mac to Llama Barn, which is an offering from GGM, the actual makers of Llama CPP. So I might have to open up a PR to add support to that. But actually on that note, how can people support NLM?

Yeah, I mean, the is you know, demoralizing as receiving bug reports can be. I think I'm overall excited about it and like, you know, give it a try and let us know if if if you want to file an issue just saying, hey, this works great, I love this, I, you know, I'd love that. Um, but definitely any thoughts you have about, you know, it would be nice if it did this as well or it would be nice if it did that as well. Think something about being a a harness for a lot of different providers is that if there's a new feature that's added, unless we get asked about it, we may not notice that little thing on the side, but if we get somebody that says, "Hey, it would be nice if it did this." If it supported this feature of that provider in specific, then we can help to get that added quickly. Um, and any of that support and there's a backlog of issues that, you know, we we tried to mark them with, you know, good first issue or or welcoming contributors. And we've also done, we're uh as an AI company, we're also AI friendly, like bring your coding agents and let's work through this together, but trying to separate the coding agent from the human. So we have certain guidelines of like using AI to write your code is great, but let's do it in a certain framework. Let's make sure that we're doing it right and that when we're talking, I am a human talking to a human. I don't want my my AI to just talk to your AI. We're not going to learn anything. We're not going to develop a community that way. So trying to help people contribute and let AI empower you to contribute sooner, but then also let us, you know, talk as humans in that process, I think is something that we definitely welcome.

Yeah, great great perspective. A lot of people I find are intimidated to jump into the open source world because they're not overly technical, but just opening up issues, whether it's it's like you said, positive or negative, just bringing awareness is always important. Um, Mitchell Hashimoto has actually done a pretty good job uh managing AI generated contributions. We're almost just like be be transparent with it. We're not wanting to shun AI code. It's still very helpful, but a m as perspective of a maintainer, you don't want to put your time and energy into trying to coach someone when that person is just an AI agent. But if you actually, like you said, having these human conversations to to get the issues to where they need to be or get the PRs to where they need to be, um, everyone wins. So it's just be transparent. Don't try to pass off work as your own when it's not. But um, still putting the time and energy into trying to improve these tools is is beneficial to everyone.

Yeah, absolutely. And I think there's been a really good transition in open source over the past 6 months where we've started to integrate frameworks that allow people to differentiate because you know a year ago there wasn't there weren't checkboxes that were saying I'm an AI agent or there wasn't a place where you could put that you used it. So then you might be like, well, I use this thing, but there's not a place where I say that this is part me and there's nowhere for me to differentiate between what I did versus what the AI did. And I think, yeah, I agree, Mitchell has done a lot of uh good work uh thinking about open source contributions. And I was just listening to an interview he did uh yesterday or two days ago with the Pragmatic Engineer uh podcast and he was he was talking about how basically, you know, he wakes up at 2 a.m. and he's thinking about vouch, you know, his open source uh, you know, contribution management tool and it's something that he's thinking very deeply about all the time. And I think it's been cool to see him being an influential voice in the community to that it's something that he thinks a lot about and because he has some very popular repositories, I think he has some very informed opinions that we're also trying to help support and and follow along with as well.

Amazing. Um, so appreciate you showing how easy it is to switch from one model to the changing a single line is is phenomenal. Um, anything else you'd like to show us today?

Yeah, let's let's hop into my newest PEP project. Um, so something that that I think I was finding as I was developing is, you know, I'm an I'm an engineer. I think about this from a very engineering standpoint. I'm a library developer. So, I'm thinking about how do I build the best library with the best code? And it can be easy to get caught in the weeds, I guess, or to guess of like, what would someone building an application appreciate? And it it's hard to like get a really tangible sense for that until you start to build something yourself. And that was something that I was challenged by a teammate uh a couple months ago of just like, hey, what are you building with this? You know, how do you how do you like, we call it, you know, dog fooding or fox fooding. How do you how do you know this is good or not until you're actually building something with it and finding the the stress points? So I was like, okay, I hear you. And thus was born uh porch songs, which is an open source project that that I built recently. And the idea of porch songs is um I like to listen to country music. And my wife has told me it's funny that you listen to country music because we live in the suburbs. You don't drive a tractor. You don't drive an F-150. Uh, but you know, I'm like, I like the the uh the message that goes along with some country songs, even though the content is completely irrelevant to my life. And as an AI engineer, I was like, wouldn't it be nice if there was a way that I could preserve the the great melodies and the meaning of these songs, but adapt them a little bit more to something more relevant for me. And that and from thus it was born porch songs. The idea being this application where you can go and you can take, you can paste in just a raw chord chart and you can edit it to make it something that you want, whether it's a silly song to sing with your kids. Uh, I should also say I play the guitar, so I wanted chord charts that I wanted to be that I could play. And I wanted the rhyme to say the same so I could sing the song and just have the words be different. And getting a rhyme scheme and writing a poem a year ago in LLM, I think would be bad at. But, you know, I did it, I did it uh with this this app the first time like a week or so ago, and I was just blown away. I was like, this song is actually good. Like the rhyme scheme is preserved and all this stuff. So anyways, uh then I was thinking about of course with this like, okay, I want to use some of the NALM features and something that we've recently been working on that's in beta right now is uh called the NLM platform at nelllm.ai and what it is is a platform that we host for you where you can manage your keys. So each of these providers, if you want to talk to OpenAI, Anthropic, etc., etc., you have to create keys on their platform and then put them on your system somewhere and that can kind of be cumbersome to deal with that. So uh we we basically have a vault here where you can upload all your keys. It it has really strong encryption so we don't actually ever see your keys. you you make one key to access your platform or your project called the NLM key, which gets encrypted and then you store that and we can't actually decrypt it and then that key which we can't decrypt, we don't have on our side is used to decrypt all the additional keys that you add. So basically you can add the keys here and it's like no one can ever see them except you and if you lose the one key that has access to it, then you have to start all over again because we we can't see it. So you can have a lot of faith in it, you know, being a a safe place to put your key basically. And then with that, you can add all these keys and then when you use it from the client side, it will track what you're using and how much. And something that like uh I think is, let me just hop over here. There are tools like open router if you I don't know if you've heard of that, but that's like an application where you can uh have a kind of similar concept where you go into their application and you can explore all the models and then you can talk to them. And so if you want to talk to the newest Quinn model, you know, you can go through open router, the one key and pipe it through um, but you know, I'm also into open source AI and self-hosting and I have a Mac mini running in my basement and so if I go on open router and I look for Nathan's Mac Mini, it's not there because it's not it doesn't go through open router. It's hosted in my own house in my basement. Uh, and this is where any LLM really shines because since the NALM platform is managing your keys and receiving your usage tokens from the client but not actually routing any data of the tokens through it, then you can talk to all your local models and still see the token usage from this hosted platform and not have any like not have to worry about how to push content through the NLM hosted platform. It's just like a sidecar that helps you manage it. And um, something that I like to try out is this is my my local my Mac Mini in my basement running LM Studio and trying out this new Quinn 3.5 model that recently was released. And I can easily add all the different models that I have keys uploaded for. And then I can go in and I already have a a chat workshop going. So, I put in uh Billy a Billy Joel song, the the piano man, which then uh I basically went through a couple iterations and I asked it to, you know, fix the chord the chords to put it in the right place and then make it uh make it be like a software engineer and then it rewrote the entire song from the perspective of a software engineer. And like I think I used I used Kimmy for a couple and I used Miniax 2.5 for some of them and I was just like amazed at how good it was. You know, like the rhyme scheme was preserved, but it's like, you know, telling this story of a software engineer that's trying to, you know, uh, improve to a project manager, but he's going to be stuck in QA forever. And uh uh it's just like it's a very funny song and how easy it was to set this all up and how easy it was to switch between all of my different models and having it all be managed in the platform and in the vault in the platform so that I don't actually have to go into my service. I don't have to open up my own source code and put in new API keys and and add new special routing to say, you know, if I'm going to the cloud, use this piece. If I'm going to local, use this piece. It all just goes through the NLM SDK and it handles it for you of like talking to each service. And then because it's connected to the NLM, I guess you can't see the pop down. That's unfortunate. Um, there is a pop down that shows you all of different providers that I'm hovering over, but you're not able to see I guess in the screen share, but basically then as soon as a new provider pops in the seam scene, you know, XYZ provider newly released, we add it to any LLM and then you don't have to do anything new on your side and then all of a sudden it's an available provider for you. So then you integrate any LLM SDK once and you don't have to worry about supporting new providers when they're available. It's just going to automatically pop in for you and and work for you as the other ones did. Uh, and it's just exciting because with this one, I was able to quickly see like, okay, this new Quinn model came out, is it any good? How does it compare to Cloud Sonnet? And because it was already, you know, supported in NLM, then all I had to do was just add a new checkbox basically. And then as soon as Open Router added it on their service, I could immediately use it and try it out and see how each of them work. And then you can like just see the differences in how each model thinks and start to think about how you need to prompt one differently or like this one is good for your task and this one isn't. And yeah, I just get really excited by it.

It's super cool. And and on the platform, one thing I noticed is you can actually set a budget per provider. So you could have, you know, just a hard cap on your open router credit so you don't completely drain all the time and then just revert back to a local model.

Yeah. Yeah. Exactly. You can you can get that set up. I think the the trying to support the self-hosted and local world. I think the years of 2026 and 2027 are going to be a lot of people doing a lot of tinkering with local models and trying to make that as easy as possible and to be able to see because one of the things that's going to happen, I I think is that is at an enterprise level, you're going to see people starting to point at self-hosted LLMs that are, you know, using these open weight models and having a dashboard where you can see that transition happening of like, okay, these people, you know, they hit their budget caps on these models and now they're starting to use the free ones that we're self-hosting. You're going to be able to see all those trends happening on this platform and then you're not going to have to worry about, you know, managing all this stuff yourself and it it tries to free you up a lot in that way. And I'll also say, you know, I I basically vibe coded this with cloud code and nelm was really easy to build on top of with an agent because there are such strong types that already exist in any that it's really well set up for the agent to go to the website, scrape the documentation, load it up in Python, and see all the different types and before even running it to say like, uhoh, this linting failed because I didn't hook this type up correctly. And all that is kind of handled for you, which makes it I think really well suited to the agentic world and like it's still it's still a decent lift for an agent to have to retool things. So NLM I think is still a really nice benefit in the world of agentic coding because it just like simplifies one thing for you that the agent doesn't have to worry about. Um, that that I think is is providing quite a bit of value.

Yeah, absolutely. A lot of value. Um, just because you mentioned cloud code. Uh, and that's well, that's one of my main tools of choice. Um, a lot of people will ask me about how it conflicts with my pro open source mentality when they're saying, "Hey, you're using this proprietary tool. What what what's the deal? Why are you doing that? Why are you just relying strictly open source?" But like, right tool for the job. Um, humanity didn't get to nuclear power without oil first. It just there's some leaps that you need to to or some steps you need to take along the way. And I honestly believe that this is one of those things where we can start setting up evals. And I've encouraged peop people listening to just set up their own evals as soon as they can where they just find tasks that they want to accomplish on a regular basis and and see which models can actually do it. And any LM would be a perfect thing to plug in there when you can just rapidly go through these different models with a list of tasks that you want to accomplish, see what's green, what's red, and what actually works because there are so many benefits to open models. Um, you mentioned the three billion parameter one. The amount of energy required to run that is so much lower than these big models that if you're concerned about the environmental impact AI is having, you want to convert as many of your tasks to these small models as possible. And people are kind of hesitant because they're not sure what's capable of. But having a tool like any LLM connected to an eval suite will give you exact granularity to what is and is not possible. And it's just it's so exciting knowing that we can actually start solving these problems through engineering efforts like any LLM.

Yeah. Yeah. Absolutely. The in the with the world of cloud code and open code and codecs and mistral vibe and all that kind of stuff. It's I think it's the same world where you don't want to lock yourself in to those, you know, those applications too soon either and making sure that you're continually trying out new things and the model choice behind it, I think is a big is a big lift. Like honestly, something that blew me away just in building porch songs is like uh I started off with Claude in Anthropic and I mean the opus and sonnet models just blew me away with how good they were and then I switched to the Kimmy K2 uh.5 and like it actually kind of shock shocked me. I was like, "This is actually like really good." And I so many things happen. You start to just like see a new model every day and you're like, "Okay, at what point do you just like pick one and stick with it?" And Kimmy K2.5 for me was another like it is still worth checking because this shocked me at how good it was. But I've tried some other open weight models that have not been they haven't met my expectations. And you look at these artificial analysis intelligence indexes and everything and they don't really tell your story, you know, they don't tell you the the benchmarks don't tell you how good they are for the thing you're building. Uh, I haven't seen a benchmark that's like how funny is it at song rewriting. Like I had to actually try it, but very quickly with just a couple different models, I could see this one doesn't do it for me. You know, I could do like a minute check to know, okay, this is not good enough for my use case. This one is. Um, so just trying things and make it easy to try things because you don't want to get boxed in too early.

Yeah. Um, some of funny evals, uh, buddy Freddy Vargas, uh, from Quotient AI, he has an eval suite where do you remember back in the day watching Pokemon have that guess that Pokemon is great out. He actually has that to LM to see how they accomplish it.

I So I run uh, GLM 4.7 locally. Um, I'm also just curious if you have because like K2 or Kimmy sounds great. Um, do you have recommendations or just your own personal opinions on which open models are good for which tasks? Where do you prefer? Um, do do you still switch between stay fluid or is there any like choices for February 26, 2026?

Yeah, I still have I I have an LM Studio running in a Mac Mini with it's a 64 gigabyte VRAM, which is it's big. It's not massive. Like I'm not running any Frontier open models with it. Um, but I've really been liking uh GPTO OSS 20B has worked really well for me kind of over a decent amount of time. Uh, the Quinn 3 model has been working really well. So, I have another thing that I deployed in my home stack called break chat and it's basically a a chat GPT clone that sits inside my house that connects to like our meal database. So whenever I like I need to meal plan or get a grocery list, I go to break chat and I'm just like, tell me like give me a recipe list for the week, you know, get me a shopping list and do all these things. And I use a local model for that. And um, the the Quinn models, they do a good job. The GPOSS model does a good job. Uh, and it's all it's all completely local, which is pretty fun.

Yeah, it's so cool. Um, I have an Obsidian vault for health related stuff, whether it's like a workout log or or just general tracking. And that's some of the information I'm not totally comfortable sharing with the proprietary labs. Like I'm I'm open book with a lot of things, but health, I don't know. It's just it's at that gray area. But being able to have GLM47 run over my thing and just say, "Hey, what is your recommendation for uh my physio exercises for today?" And it can just analyze what I've done recently and just give a recommendation. It's just an extra tool in your toolkit to give you a little bit of insight. Uh, maybe can help you with your grocery list so you can make healthier meals. Um, but there's just so many little aspects of life that can improve if you're willing to pair with AI. And if you're not comfortable, there are local models, so data doesn't go anywhere.

Yeah. I think that one thing I've seen a lot when you get a local model and they want to try is they're like they ask some really specific informational question like, you know, tell me who was the president in 1985 or something. And you're like checking all these knowledge things. But when it's a small model, it's being optimized a lot of times for tool use, like you're saying, of like it doesn't know all these things about the the history, but it's good at like calling tool, retrieving information, making judgment calls based on that. I think that's a lot of the way that they're being optimized. So for a lot of the tasks that you want to accomplish locally, these local models are actually like pretty good at just accomplishing tasks. Like they might struggle at rewriting a song in a funny way uh of something that you can run on a small computer at home, but they'll do a pretty good job of like doing a Google search, returning the results, you know, executing tasks for you. I think they're they're perfectly equipped to take care of.

Yeah. Summarizing summarizing text, um, there's a bunch of tasks where you don't necessarily need to send it away to an API. You just have it done right on your machine.

Yeah. Yeah. Absolutely.

One question I have, um, I'm a huge fan of Go. One of my favorite languages. Um, but it's not always the most popular, especially in the AI world. Why did you decide to make a library for that one?

Uh, that was somebody else on the team. I am not a huge Go person. I'm more of a a Rust person. Uh, but there's we have a different project on the team called uh called MCPD. It's a a tool that helps you manage all your MCP servers and that one is written in Go. And we just decided it was a good, you know, second choice to be able to help people integrate with these model providers. Um, but we're planning to make them in pretty much every language to help give more options. Like there's another tool I work on a personal project called Agent of Empires and it's basically just like a session manager for coding agents and in that, that one's written in Rust and I want to be able to connect that to the platform that I just showed to be able to like track the token usage of my coding agents when they're all working and stuff like that. So, we want to put it in every language just to make it as easy as possible for everybody to have that choice and and know what's getting used in their model stack.

Asian Empire is a phenomenal name. Great choice.

Yeah, I chuckle every time that I say it.

Excellent. N this was a blast. Thank you so much for showing us about NLM. I legitimately see a ton of value in it and I hope everyone listening at least gives it a shot. Like just being able to experiment with new models is such a valuable process to go through in this day and age. Um, before I let you go, is there anything you want the audience to know?

Yeah, I mean, Misilla AI loves the community and contribution and like we said earlier with coding agents. It's never been easier to get up and started within with making a contribution on GitHub. You can basically just ask cloud code or anything like, how do I solve this? How do I make a contribution? And it will help teach you how to do this. And you know, there's a lot of work that we can all do together to help build these things. And we're open for contributions. you know, give them give them a try, see how they work, uh, and let's help promote the opensource openweight AI ecosystem together.

Thank you for this my conversation with Nathan Break. I love talking to Nathan and just seeing him light up talking about open source and open source AI. It's so important that we start exploring these tools and supporting these offerings because we don't want to be renting AI forever. We want to be able to take control of our data, do things that will help the environment, and move AI forward in a sustainable, safe way. Open source AI is the key and tools like any LLM are going to enable that for everybody. So what I ask of you is to make your own eval suite. Find some tasks you ask AI to do, some successes, some failures and just put them all into a document and then use your coding tool of choice to set up a script so that any LLM can test out different models for these different tasks and you can get an output to see which one accomplishes what. Because the better model you use might be the smallest model because it will save you cost. It will save energy and it'll still result in the output that you're looking for. We don't need to use the mass models for everything. And when you have tools like NLM, it's a lot easier. Thank you for watching and I'll see you next week.