Transcription
It feels like every single day we're getting a brand new open-source project on GitHub that changes the way we can use Cloud Code. So, to help cut through all of the noise, in today's video, I'm going to show you five of my favorite open-source projects that have just come out in the last month. Some of these are heavy hitters you've probably seen before, but I promise you there's a couple that will surprise you.
So, let's kick this off with the biggest player of the bunch, and that is Auto Research by Carpathy. This thing is up to almost 60,000 stars, and it came out just 3 weeks ago. Essentially, this repo is a machine learning algorithm in a box. And you've probably already seen some content about it already, but I wanted to go over it and kind of explain how it works and how you can get the most out of it because it can be a little confusing if you don't understand, you know, kind of machine learning fundamentals.
Now, how Auto Research works in a nutshell is it allows you to point Cloud Code at some sort of task and have it run automatic tests over and over and over and over again to self-improve. Every time it gets better, it commits that improvement. And every time it gets worse, it discards it. And the idea is with all this automated trial and error, over time you get a better result of whatever you're trying to do. And the results people have had from using Auto Research is pretty impressive.
So this is a tweet from Toby, who is the CEO of Shopify. And he used Auto Research on an internal model inside of Shopify, a rather small one. It's just a 0.8 billion parameter model. And after using Auto Research and having it run 8 hours and do 37 experiments on its own automatically, it became 19% more efficient. So this is really cool stuff.
Now, if you want to use Auto Research yourself, there's a few things you need to understand. You need to understand how the repo is built. There's really three core files. And then you need to understand the sort of use cases that make sense cuz we can't just point it at whatever we want and expect it to work. Also, we kind of need to understand the basics of machine learning for that to all make sense.
So, here's your 30-second machine learning crash course. So, the idea is you point a neural network, you point a computer at a problem. Let's say you want it to see how well it does at recognizing cats in a photo. The computer attempts to recognize the cat and then it gets a score of how well it did. Based on its score, it then adjusts what are called weights. These are internal essentially parameters that change the sort of output it gets. And then it does this again. Tries to recognize the photo of the cat, gets a score, adjust the weights accordingly again and again and again for millions, billions of times. Okay, so it's just attempting to do a problem, gets a score, adjust weights. That's essentially how this works, right?
The problem scenario for when we're using the Auto Research is defined in the program.md file. This is what you edit. So this is where you put the constraints. This is what you're saying you're trying to do, right? In the last example was recognizing cats in photos. This is whatever you want, right? This is what you touch.
Now, when Auto Research runs, it doesn't have weights necessarily, but what it does have is a train.py file. This is what the Cloud Code, the large language model you're using will edit. So, it doesn't have a trillion weights. It just has this code. So, based on its run for that particular time on the score it gets with Auto Research, it's going to adjust this, right? Think of this as the weight.
And then we also have the prepare.py file. We don't really touch this, right? This is what was actually built by Karpathy. This is like essentially the foundational, you know, code for this thing even running in the first place.
Now, for this to work, it needs something to score it on, right? It needs some sort of like binary metric that it's actually working when we test it each and every loop, right? It can't have a subjective answer after every run. It can't be a situation where like you hop in and say, "Oh, that was good. That wasn't." Right? So if you're trying to run the Auto Research and it's like a subjective or requires live data some sort of output like that right, "Hey, is this social media content good? Was this a well, you know, written cold email? Anything with creative writing?" Right? This is you can't really run this for things like that, right? Because how is the large language model supposed to know if it's good? It's completely subjective. That's not great for this.
Versus what this is really good at is, you know, any sort of like, you know, discrete numbers, right? A number. You know, "I'm trying to create a program where the Python script runs faster." Okay, that's easy to see if it actually worked or not, right? Any sort of like prompt optimization, right? Anything that has to do with like skills, like Cloud Code skills, it's actually going to do well in: "Did the skill pass or did it fail?" Right? "Did it work or did it not?" And then like system prompts. "Is the output that we're getting on each run matching the expected format?" It's a yes or no question, right? So if the machine can't score it with a yes or no, this is kind of nonsense. There's no reason to use the Auto Research loop. So just understand going in what makes sense for you and what doesn't.
And so with all that in mind, the loop should make a little more sense, right? It runs, it gets some sort of output, and then it gets a score. Based on the score, it will do one of two things. If the score got worse, right, it's a worse score, we do a git reset. If it was an improvement, yes, we commit. We save the code changes. And then from there we do another run, right? It thinks of something to try. Then it edits the train.py file, right? That's the equivalent of adjusting the weights in a proper machine learning setup. Then it does another run. It gets a score. Did it improve? Did it not? All right, run it again. And just keeps doing it over and over and over again. That's the Karpathy loop. And obviously every single run isn't going to lead to an improvement. As you can see here, the first example on the repo, 83 experiments, 15 kept improvement. So really cool repo. Definitely check it out if you haven't. Just understand it's really going to be a function of what sort of use case you have. This isn't a one-size-fits-all thing.
So before we jump into repo number two, just want to give a quick plug for my Cloud Code Masterclass. This just came out a couple weeks ago and it is the number one place to go from zero to AI dev, especially if you don't come from a technical background. And I update this thing every week since its release. We've already added nearly 2 hours of additional content. We've updated 20 plus modules and yesterday I just added my entire Twitter research engine. So, tons of content there. Definitely check it out. There's a link to that in the pinned comment.
And if that's just too much, right? Hey, you're just new to AI, you're just getting started, I also have the free Chase AI community. There's a link to that in the description.
Now, repo number two is OpenSpace. And this thing is all about self-improving and self-evolving skills. Now, this just came out 4 days ago. It's at 1.7K stars, and it's from HCUs. This is the Hong Kong Data Intelligence Lab. And if you don't know these guys, they are the authors of a ton of really cool open-source projects like Lightra, Nanobot, Rag Anything, Deep Code, AI Trader, Seite, Anything, right? So these guys again are kind of titans in the space. So when they come out with something, it's worth it to take a look.
So the general idea of OpenSpace is that it's essentially an MCP that tracks your skill usage when you're using it for a specific task and it does quality monitoring. And from there, after tracking the skill performance, it kind of puts it in one of three buckets: either autofix, autoimprove, or autolearn. Autofix means, "Hey, it's identified a skill that's just like not working at all." Autoimprove means, "Hey, the skill worked, but we can do much better." And then autolearn pretty much says, "Hey, like this thing is as good as it gets. Like, let's capture this and make sure we don't play around with it anymore."
And while there is an upfront cost to having this sort of quality monitoring at first because you're using the MCP server, OpenSpace is telling us that in the aggregate, there's actually a lot of token efficiency to be had here, right? Because we're no longer going to be repeating work because the skills weren't working as they should. And it says tasks get cheaper. In that ultimately from their tests, these sort of agents using the improved skills used 46% fewer tokens on real-world tasks. They also put an actual dollar amount to that. I think it was like $11,000, but that was based on their own internal metric known as GDP. So take that with a grain of salt.
That being said, they do go into detail in terms of this benchmark, right? They say, you know, they use 220 real-world professional tasks spanning 44 occupations and that ultimately using this system versus like the standard system they use GPT-3.5 Turbo for all of these, right? It generated 4.2 times higher income, 73% value capture, and then 70% quality versus 40% quality on the baseline. So by their numbers, this sort of like auto-improving skills system does work.
And one of the big showcases they use is like one of these like superworld monitor things, right? Where you have maps and you have, you know, video streams and all these analytics coming into one dashboard. And so what they're claiming is by using OpenSpace, their AI system was able to create this. It was able to create 60 skills in the process after starting from just six initial skills. So, by this self-improving system, it then added 54 skills in the process, right? And they have a whole tracking system. So, interesting stuff. Again, this just came out like 4 days ago. And if you're someone who's like really deep into skills and, you know, always trying to figure out ways to self-improve it, definitely take a look.
Now, repo number three is also going to be from Huds. And if you've been following my page, you've seen my work on this, and that is CLI Anything, Command Line Interface, Anything. So, if you've been paying attention to sort of the Cloud Code ecosystem lately, you know there's been a huge shift from MCPs to CLIs because CLIs tend to be more effective and more efficient. And this tool allows us to turn anything into a CLI if it's an open-source project and we can actually see its inner workings.
Now, this one came out in the beginning of March. It's got about 24,000 stars. And the nice thing about this is it's so easy to use. Like obviously it's sort of, you know, a two-line install for the plugin and the tool itself. But from there all you have to do is run a single command where again, you point it at some sort of open-source project and then it runs the entire pipeline, right? Where it analyzes the code, does tests, documents it, and then publishes it as an actual tool for Cloud Code to use.
And they've proven the system works by pointing it at a number of different projects. Things like or Blender and Audacity and OBS and Zoom, right? So, they've already done this and if you go through the repo, you can actually take what they've already built and use that yourself. Or, and I'll put a link to that above, like we did, we showed how you can use it and we use it on draw.io and it was very simple and very effective. And the nice thing is after you point it at it the first time and it creates it, you can continue to refine it and add additional functionalities to the CLI if on the first pass it didn't give you what you wanted.
And the biggest value with CLI Anything is continuing to close that agent software gap. Like as good as Cloud Code is, there's certain things that it just can't do. Like it can't interact with certain software and it would just make our lives so much easier if we can do everything through the terminal and this fixes that.
Now, repo number four is Cloud Code Peers. This just came out last week. It's just over a thousand stars. So it's a smaller one, but it's an interesting one. And what it does is it lets your Cloud Code instances find and talk to one another. So now when you have multiple terminals going on, they're not necessarily in a vacuum anymore. It's almost like they're agent teams that you control and they can actually speak to one another.
So it works via an MCP server and an SQLite database and it essentially auto-launches when the first session starts and once you start additional sessions, essentially a summary of what you've been talking about gets pushed to all the additional sessions that you spawn.
Now, why should we even care about that repo? What sort of use cases make sense? But when I got introduced to that repo, I immediately thought of this blog that came out from Anthropic on the 24th and it's all about long-running application development and figuring out these sort of harnesses for creating things. Now, this is a super cool article. I'll link it down below. And essentially what it's about is they say, "Hey, like we're trying to get Cloud Code to create two things that it struggles with, you know, when you're just working in one session. That's things like front-end design, right? And then also like creating a video game, right? If you just work in one session and you ask it to do something complicated like that and then ask it to grade itself, it tends to do really poorly."
So this article is essentially saying like, "Hey, if we're going to create these complicated projects and it's going to take both a lot of subjective grading and iteration. How do we do that?" Well, they propose that what you need to do is you need to basically take your Cloud Code sessions and break it up into three parts, right? You need someone to plan, you need someone to execute, and you need someone to evaluate because the problem with Cloud Code is it is a poor evaluator of its own work. It kind of hypes itself up.
So, if you're looking to create some sort of, you know, link between multiple sessions and a long-running harness and have one be the creator and one be the evaluator, well, better yet, let's have them actually talk to one another. So, something like this Cloud Code Peers in that sort of harness setup, I think, would make a lot of sense, right? Because you could have the actual executor, the actual creator of the code in one session and then you could have the second session be the evaluator and you could very easily have them talk back and forth and that harness almost creates itself at that point. So something to think about. I think there's a lot of potential use cases with this sort of technology.
And last but not least, repo number five is the Google Workspace CLI. Now this is essentially an official non-official Google product. Right? It's not officially endorsed by Google, but this was created by Google developers. And what this allows us to do is give Cloud Code access to your entire Google suite.
Now, two ways you can go about this. You can kind of go nuts and give it access to everything like Cloud Code controls your whole email system. Or, like I break down in the video above, hey, here's how we can sort of like sandbox this thing so we can get the most out of the Google Workspace CLI without necessarily exposing it to everything.
Now, this is obviously a huge value add if you're using Cloud Code as sort of a personal assistant, right? Giving it all of your Google stuff is super important. Although the one thing I will point you at is sort of the skills setup they have here because they have a bajillion bajillion skills. So I think before you go through the whole install with all this, you should take some time to, I would say, clone this repo with Cloud Code and then have it walk you through what skills make sense for you because again, it isn't an all or nothing thing if you use this repo, right? You can maybe just have it have access to your Drive. Or maybe you just give it access to Docs. Or maybe you give it a separate email and you give it access to your Drive through a shared folder, right? Sort of that sandboxing thing. So I definitely suggest checking this out. Very nifty tool because again, anytime we can give Cloud Code more access to these sort of productivity tools, the greater it is for us, right? Especially especially in that personal assistant type use case.
And the one last thing I'll mention with it is because obviously the first thing that comes up was like, "Oh, I don't want to give Cloud Code access to my Gmail because what about prompt injections?" Well, they have a number of skills as well as a whole thing to do with Model Armor, which essentially is Google's built-in prompt injection sort of armor, right? Essentially, it will scan things and see, "Hey, does this look like it would be a prompt injection?" before it actually goes to Cloud Code. And you can set it up to where it just flags it or it just like completely gets rid of it entirely. So there are certain safeguards in place from the scarier, you know, situations of giving Cloud Code access to things like email.
So those were my five favorite open-source projects on GitHub that have come out in the last month that I think can change or at least enhance how you use Cloud Code on a day-to-day basis. So make sure you check them out. There's so much cool stuff coming out that is open-source and free to use these days. And it's it's kind of hard to stay on top of it all. So let me know in the comments what you thought. As always, check out Chase AI Plus if you're looking to get your hands on the Cloud Code Masterclass.