Transcription
You haven't been paying attention to what is happening to the world of AI. Now is the moment because we are entering the sci-fi moment.
Okay, this is completely incredible. What we can do today couldn't be possible to do just a few weeks ago. Now, why I'm saying this and why I'm so pumped about this? Because I'm not just talking about those uh frontier model AI from those large corporations. It's also what is possible to do with your local AI.
Okay, locally I meaning from running from your mobile phone, on your computer. I can run almost anywhere. Okay, so there's no excuses and it's free for everybody. And each one of us can have a local AI that can work for you 24/7.
Now, what is the big difference that happened in the last few weeks? That they came out with a few really agentic models because the demand was there. In the sense that Hermes came out, open code came out, open came out. Those systems require an LLM that can actually use tools, which is different than just chatting with AI. Okay, you chat with AI, it's okay, we're just thinking together. Using tools means, meaning you tell AI to actually take action within your computer.
Okay? And action can be, for example, do a research, go on the browser, do some, do some work outside, and write email, check my email, write on the calendar, you know, write code, write us uh a video game for me. Okay, this action that needs to to use tools to be able to do it. And there is many things that can be done. I'm just giving you uh some examples so you understand the difference between chatting and actually having an an agent that can take action within your system.
Now, the two kinds of AI uh kind of AI models that need to run those systems is different. So in the last month uh we had a few of those coming out and those are amazing. Now, it all depends on what kind of hardware you have. You need to decide which model to use. Like I said, from they can run from a mobile phone to really powerful machines. From my system, and I have the Asus GX10, which is the equivalent of exactly the same hardware of the Nvidia DGX Spark.
Okay, those are micro supercomputers that can, it's just super small that can run on, stay on your desktop. And they have 128 gigabytes of RAM. That means they can upload really large models and work at a decent speed. Now, there are better systems, of course, but for the price that I paid, I'm pretty happy about this solution. Really solid build for working with AI. So no compromise there.
What I've been running on this machine is the Gemma 4 26 billion A4B. So why this model? Because it runs at a speed that is uh good enough for me to interact with. Now, I want to, before I go and explain this this speed, this token per second generation, I want to say that yesterday I installed the Quen 3.6 35 billion A3B on uh for running Hermes. Why I I did this? Because when it came out, it first came out the Gemma 4, then a couple of weeks later, I think, or something like this, came out this Quen uh model. So I started with Gemma. I was really happy. The token per second was decent. So I said, let me try Quen. I tried. I said, okay, I don't see much difference. I'm going to go back to Gemma. And uh because I wanted to understand fully Gemma on long term. And then yesterday I decided, okay, let's give a shot to this Quen 3.6 and see how it goes.
So, first of all, understand how this technology works. While they are German is 26 billion, there are only 4 billion parameters active at at the time. So it feels like working with a 4 billion model. So the speed is connected to that kind of size model. And in Gemma, we have 3 billion parameters active at a time. So what is happening with Quen? We have this uh because just is less parameters active at the time, we have a faster token per second generation. So it feels a little bit faster. Now, in general, the the peak for those Gemma and Quen are pretty similar. Okay, Quen is a little bit faster, but they move always between 30 and 60 token per second. Okay, sometimes as soon as you start the chat, maybe it jumps higher, but they are moving within the range on the average. It depends how long is your prompt, how long is your context window, and what kind of complex task, how many tools are using at the same time. So there's a lot of elements that can change the token generation, but most of the time you are within the range and under uh 30 token per second. We as a human, we start feel like frustration because it starts to become too slow. When you work with AI as a human AI interaction, you want to stay over 30. Both of those models that always stay over 30.
Now, if you are not working together with the AI, so you have a larger model that is much slower, and what you do, you delegate some work and you allow this to work in the background. When it's done, it gives you the output. Okay, so it's not the end of the world. But there are more like more powerful for for coding, for example, and they are much slower, but you delegate and they do the job and then come back. And you delegate sometimes you do through the the agent itself. So, for example, my Hermes can control open code. Open code is a harness to write coding, like a Codex, like cloud code. Okay, same thing, but this one is open source and you can add whatever model.
So what is happening? I can ask Hermes, look, I I want to write this code. I explain everything. Okay. Then Hermes, what it does, delegate to open code and allow this to happen. Every now and then go there and check and make it, make sure that this the system is working. This is how I've been using this. When at the moment is, I'm I don't know, yesterday I I I started using it and what I saw, it's, I don't know, it felt better than Gemma. Now, of course, those models are really different, depend what you do and change what they can do. Okay, it's in in the sense, one is better for something, one is better for something else. But and uh there are some limits also. This Quen is only text. So it can't read any images. It can't do any audio. It's just text.
But that doesn't mean that you can't connect to other AI. For example, when I work on Hermes on my mobile phone, I don't like to type. Okay, just send audio messages. So I send audio messages. This message gets sent to Whisper, that is another agent. This agent transcribes the audio into text. Send the text to Hermes. Hermes receives this text and uh replies to the text. So the whole process is pretty fast. So you don't feel the the need to have an AI to actually understand the audio. Most of the will do through the Whisper.
Uh, but yesterday I I wanted to, you know, create an image for this AI. And they said, okay, I can't make me images. Okay, good. But I own this uh GX10 which can make images. So go into the GX10, install whatever software is needed, configure this software, and download the model that is needed to run it. And then give me this image. And it did it. Okay. So doing something like this. Okay, I never tested with Gemma, but I wanted to test and understand this Quen. So I asked something really difficult to do. Okay, so go inside a different machine. So uh Hermes run on my Mac Mini, and I use GX10 only for AI. So there's no other things than AI there. So Hermes found that machine. I mean, already had the settings, right? But he found that machine. He went there, check what kind of software, download the software, configure the software, download the model and configure the model to work with the software, prompt the software, get the image, copy the image, bring it to uh the Mac Mini and post it on uh on Telegram. That's a lot of work. Okay, it's a lot of work. It took some time, okay, because it was trying to understand what's going on and so on. So there was a lot of understanding because he didn't know what what was present inside the machine. Didn't know the soft. So had to install a skill to use the software. So there was a lot of things to do and he did it.
Okay, can you understand this? He did it. A local model that run 100% on my machine was able to solve a problem that I personally I couldn't do. I have to be, I have to admit. Okay, it would be, I mean, I can do, but you need to do research, understand what kind of software, how it works, how, you know, there's a lot of of learning. Okay, so I would take half a day probably to get the same result that the Hermes it took some time, I think 10, 15 minutes. So it was a lot of work. Think about this, a zoom, you know, that works, works, and try, test. So he never gave up. That is the. Okay, it didn't give up on finding the solution. I don't know if you can understand how huge this is. Okay, so the idea of not stopping until building the the solution is incredible.
Now, I want to be clear, it's there is a gap and a difference between those local models, those frontier models. We talk about trillion parameters differences. Okay. So it's a lot of difference, but there are ways that we need to to work with this local model that makes h that in some close this gap. But at the same time, uh by understanding the limits, then we change the way that we work and we overcome some of those limits. Some of the limits are not possible to pass. Okay. There is a difference between, you know, billions and trillions invested in building those frontier models and those ones that are much cheaper. There is a difference. But so let's understand what uh what's the challenge and how to how we can go around.
So the problem in here is the intelligence per token. Okay, we need to understand this concept. So every token are not the same. Okay, a token generated for a smaller model doesn't have the same intelligence that come from a frontier model. So by understanding this principle, now we can go into what does it mean in the sense, how can I use more tokens to achieve the same result that a large language model of those frontier models can do with fewer tokens. In most scenarios, the answer is yes. Okay, that is the point. So you need to understand that you will take more time to achieve the same thing.
Now, how do we deal with this more time? It's we need to understand that, okay, it's just not just produce more tokens. No, there must be a strategy on how we build this token. So in the case of uh of a software, for example, on uh on those frontier models, if is a software is simple, you can even one with one single shot build this software. So you ask AI, for example, to uh to build a Tetris. Okay, you go on Codex, GPT 5.5, one of the best models we have out there. And think about it's just $20 a month to have to do what I'm telling you now. So you go into Codex and you say, write me a Tetris game, which whatever graphic, you give some information. You ask this, boom, and it does. Okay, and now you have this game working and so on.
Now, if you go the same with Gemma or Quen, what is happening here is you ask for this game, it comes out the game, you see it, but maybe it doesn't work for some, whatever reason. Okay, or maybe there is it works, but then there is, I don't know, sound, or there is no way to restart it once it dies, or whatever. Okay, there is something that is not perfect. So what you need to do is to iterate with this software and improve it. What I did, I made a Codex look at what was uh Quen 3.6 doing while working on the code. Okay, so I had this spy looking at what it was doing. So I could have an external uh verifier to understand what's going on with this software and say, okay, how can we improve the strategy to make this better?
Now, I'm going to give you just a little elements, but it's much more complex. Uh, there is a Substack article link in the description where you can have much more information. Uh, but here, it's it's too complicated to explain and I want to be clear and simple in this video. So, uh, what you need to do basically is to have those models work inside a loop. Okay, it's not just one shot, but you should ask first to implement narrowly. Meaning, it's don't build the whole thing at once. Start to to divide things in elements because it's going to be much easier to work on elements.
So what what you do? You write first the specification. Okay, you do the specification. No, now you know what kind of software, you agree on what to do, what elements should be present in this software. So you didn't even write any code, but you with AI now you agree on what is needed inside the code. Then you say, okay, now do this in blocks, not at once. Okay, then don't even think that is done, but review adversarially. Okay, so go into the system and review and try to find what is broken, what is not working, and uh if what we agreed is actually being implemented correctly and so on. Then of course, patch it, but patch narrowly. So just go into one per part at a time. Don't fix the whole thing at once. And then verify deterministically. Meaning, I don't care that you look at the code and you say, I don't know, it looks good. Okay, that is not an answer. Open the browser, load up the game, okay, and then test it. Play the play with the game. Let me know if actually works. Okay, do does the thing move, okay, or not? Can you restart the game? Can you see the score of the that I'm making? So that is the deterministic verification.
So working in loops is possible to achieve. Now, I don't want to say the same result. Okay. But we can achieve a really good result in a simple system like this kind of game. We can basically do the game, the weather, our dream game, our dream Tetris, our dream Snake, and so on. Okay, it's possible. If you want to do something more complex, need you need to architect it correctly. So it's not just more tokens, it's tokens within a loop, within uh framework, and that is what is important to understand.
Now, if you are not doing coding, but you're chatting with AI, you're working with the whatever you're working on on a project or whatever you are strategizing and so on. What is important to understand is the because they have less uh parameters. Okay, we're talking about 26 billion or 35 billion in the case of Quen. They have versus trillions. We're talking about trillions of parameters. So the gap is huge.
Now, why those uh models can perform so well even with little uh parameters is because they are really selecting on what they have been trained on. Okay, they're being trained on really high quality data, no noise, really, really important data to be agentic and and use your software and do, you know, the most important things that we as a human want to do. So it's been condensed into that little knowledge of covering 80% of whatever we want to do. I'm now say 80% just to give a number, but so you can understand a lot of things that we commonly do is present there. If we want to go outside what is common, then it's not going to be present there, but it's going to be present in those large language models.
Now, if we understand this, okay, we can also say, okay, it's not present there, but then doesn't mean that we can't give it. Okay, so bringing, I call it awareness. Okay, so if we can build, give to those smaller models the level of awareness needed to actually address that problem, the task, whatever it is, then everything changes. Now, it's true that the AI doesn't have experience and knowledge about that specific thing, but I give it. So how do I give it? Sometimes, most of the time, is our data. Okay, we need to share with those agents our data so they can actually think within our world. And this is the same that happened with large language models. So if they don't know who we are, of course, whatever output they give, they might be in line or not in line with us.
So if we give to those AI, I call it our creative DNA, who we are, and it's not just about, you know, your, it's not your CV, okay, your success, uh, or your, or just your project, or your business plan, but it's also, I call it anti-museum, what are your wrong turns, your scars, your problems, your values, all the elements that define who you really are. Now, has those elements to understand you and to work within your world to be in line with you. Now, the output is more correct in your sense. And I want to make it clear, this is when you work with strategy and documents and so on. When you work with uh writing code, AI doesn't need to know your values. Okay, it doesn't need to know your scars or your journey. So that would be noise. In the case of writing code, but in the case of working with you on a strategy and document or whatever, it's those are not, this is not noise, this is the compass. Okay, this is the compass that tells AI how to interpret reality.
So basically, you're sharing your philosophy to understand how to read the data. Okay, philosophy is uh is kind of the how we interpret reality, how we look at the world. It's like a lens that looks the world in a specific way. Now, we give the AI the same lens that we have. So AI looks at things in in our same direction. And again, this is not just for small models, but also for those large models from those corporations. So we need to give awareness. Sometimes, like I said, it's our data, but a lot of time is also data that is doesn't belong to us, but we can find it on the internet. So run a research.
So before, for example, now you go back to the video game. So before writing Tetris, okay, you could say, do and do a research so you understand exactly what Tetris is, how it works, how it should the game look like and so on. So you could do a research. Now has the awareness needed to work better on the code.
Now, in the case of code, we need to also understand this element that most of the time we don't need to write the code from scratch. There's a lot of open source code already there. So what AI should do? First, do a research. Look into the open source code that is under the MIT license, for example. Meaning you can use it for whatever, even commercial use. Now, go there, find the code elements. Okay, put it together. Now, it's all about like a play Lego. Okay, you need to just put it to together. When I say just, it's not that simple, but again, you don't need to write everything. Meaning the code that you're getting is already well road tested. So it's more reliable than the one that you would write. Okay, it's it's been used by, I know, thousands of people and so on. So being perfected over time. Now, you connect those elements together. You save in time. You save in to you have a better output. And again, this is also the best strategy for those frontier model AI because there's no reason to write to reinvent the wheel. Okay, if it's already there, let's use it. It's open source, great, let's use it.
So that is the the strategy. So if you understand those um elements, giving awareness, do research, don't write the entire code, but search for what is already available, have those loops of writing, testing, and verifying and so on. So now we have even with those small models, we can achieve incredible.
What is the difference then? Why it still makes sense in some cases using those frontier models? So like I said, it's intelligence per token. Now, the intelligence is going to always been limited. Why? In most scenarios, we don't need the top intelligence. We have going to have a gap of intelligence. Now, this is again something that we need to understand because there is a sort of a misunderstanding of what is intelligence because even the large language model, I I call it like a PhD level idiot. Why is that? Because they actually, they're not intelligent. They have the knowledge of PhD level and and an intelligence that is almost like an idiot. Now, with that kind of knowledge, it's possible to achieve a lot of things. Okay, you're not intelligent, but you have those knowledge and therefore you can still do amazing. In some scenarios, like coding, they are not idiots anymore. Okay, they're kind of better than idiots. But a lot of time, you can see that they are not intelligent. Okay, they they completely lost.
Now, the point here is, we are the intelligence. Okay, we have intelligence, we have intuition, we have things that they can't even understand. So now it's about us closing the gap. So while feels like sometimes possible undoable, even though it's a mistake, delegating intelligence to those frontier models. So you make this AI thinking for you in somehow. On those local models, you can't really allow them to think for you. You need to do the thinking.
Okay. So those elements of strategizing first, no, before writing a code, do a research, then plan, write the uh the specs of the software, and then do all those loops to build this is is you. Okay, this is you knowing how to use those larger models at the best scenario. Now, you go to the large language models, they, you don't, you don't use this strategy. What you do, you just say, ah, write, write me this name, and so you are, you delegate the intelligence onto this model. So the chance of having something that works is higher because they're more intelligent, but at the end, it's still not good enough. So in some scenarios, it's still better to implement the strategy that you have on those small uh language models into the frontier models. So it feels like you could do even without, but it's uh it's a feeling. Okay, which is uh different from reality. And uh why I'm I'm talking about feeling? Because there is, we as a human, we use feelings to navigate this. And the feeling of AI, this local AI can do something similar to the large language models from those frontier uh systems, it's uh it feels like they can do the same. Okay. So there is a lot of feelings that we use to navigate this scenario. While if you actually do a AB, you can see the difference.
I want to give you an example. Okay. So for people that are into audio, like myself, you can see my instruments here. It's it's a little bit like working with um uh MP3 or high resolution audio. Okay. Professional audio format versus the MP3 consumer, the one that you download from the internet and so on. So the MP3, when you hear it, it feels good. Okay. On the background, it's perfect. You don't need, you know, high definition. Most people don't even have the speakers that are good enough to to hear the difference. And some people have those speakers and they don't position well enough to hear the difference. But if you are into high quality, then you position the speakers, you and then you do AB with MP3 and professional audio format, and you can definitely feel the difference. Okay. And that is similar of what is happening with this AI.
So the local model, even though it's good, amazing, you can achieve a lot. If you compare to what can be done with those frontier models, you will see the difference. In some scenarios, like I said, background music doesn't matter. Okay, you don't, you your brain is not attentive to this, pay attention to this. So what is the same thing when you, for example, write an email? You're not thinking about, I need to write the best email ever. Okay, I said, the best email I ever written. No, you all you want is reply to this email. Okay, then you check and you send it. That's it. Okay, simple. You need a summary. You don't need this amazing thing. You need a researcher. You don't need always to have the most important research of your life. And be clear that even the frontier models cannot do this amazing top things. So we are still far away from actually achieving with one single shot an an incredible output.
So I want to say this thing to just bring back to reality. Okay. I love those u local models. I use them every single day. Okay. So it's part of the way that I work. I still have also subscriptions with those frontier models. And why I do this now? Now, this is the the moment of truth, why this is needed. Okay, let's understand the trajectory where we are at the moment. We have the luxury, let's say, to still be able to use those frontier models. Okay, because they are sub subsidize, I don't know the words. They are basically giving us at a price that is cheaper than what it costs for them. Okay. So they're giving us a free token, let's say, inside a subscription, which allows us to build so much while uh at their cost. Okay. Because they want to capture we as as clients, they want to learn from us by extracting data and so on. So they're doing this uh is a tradeoff. Okay.
In my scenario, what I do, I do some elements with local and some other with uh this frontier model. Like I said, I had GPT 5.5 monitoring Gemma 3.6. So it could see what kind of error, what kind of problem, what kind of gap were missing. So now I create a protocol to allow this Gem to, sorry, this Quen 3.6 to work at a higher level. And I couldn't do it by by myself. Okay, I it would be impossible to do it. Time, energy, effort, knowledge will be impossible. So now, in a couple of sessions where Gemma was coding and having this large language model monitoring, I learned so much. I improved the protocols and now there is a skill designed to allow Quen to work at a better level. So think about how we can leverage that knowledge now. Now that we have available, we should leverage on it. Okay, this is my strategy. Now, it's not for of course for everybody. Some people will have different needs and so on.
So on building, let's say, you need to build what I'm doing. Okay. Now, let me explain what I'm doing. I'm building this. It's called the Resonant OS. This is uh a system that allows all those agents to work within. Okay. So it's a framework where I have my high-level thinking agent that I can trust. Okay. One single AI that understands me totally. It's I can totally trust. Has a really high-level knowledge and therefore I can strategize with that one. That is the, it's called Augmentor, someone that augments me and uh allows me to do more. So that AI is there, part of Resonant OS. Now, on the workspace, I can load up, for example, Air dashboard. So you can load up there and now they can work together. So the Augmentor can delegate to Air to do agentic work. Now, Hermes itself can subdelegate, can have sub-agents, can delegate to something else. But the point here is, I have an AI that I can trust. And I have her open code, whatever what's the name, Paperclip, and so on, running within those framework.
So, this is what I'm building. This is what I'm building with the community because this is owned by the community. Going to explain why it's owned by the community. But why I'm I'm doing this? I'm to build this, I'm not using the small local models. Why? Because I don't, I I don't, I want to leverage on the time that I have and the cost of those free tokens to build the infrastructure with the minimum energy that I need. Okay, so I'm I'm building the big part with it. And and as long as it's possible, I'm going to use those uh frontier models to build it. But Then I have now a structure built where those local models can actually work with it. Okay. So they can start improving, changing parts for a lot of works. Once once it's been organized and and created into modules. Now, those uh larger models can work within those smaller parts of the code without any problem. So this is how I'm doing it.
So regarding the this Resonant OS and the Resonant DAO, what is what is this? So I, if you're like me, probably you're realizing that we are and working with AI. Okay, you're realizing that what is happening in the world of AI is pretty insane. And they are building AI to replace us. My idea here is to, okay, let's leverage their their technology to build the alternative. Okay, the alternative is this, decentralized autonomous organization. It's called the Resonant DAO. H we are building an alternative economy system that is built by with their technology to for us. Okay, where we are at the center. And leveraging this technology is not as a enemy of the system, but as a something that can augment us, not replace us. That's why we have the Augmentor. That is the philosophy. I don't want to go too too deep, but hopefully it makes sense. We are building the alternative. The DAO is decentralized and owned by the community. And Resonant OS is the software owned by the DAO, therefore owned by the community. And this is what we're building.
Now, if you believe in something like this, if you believe you can change the world, okay, if you believe this is needed, join the community. Link in the description. Like and subscribe to this video. I hopefully you stay until the end. And if you stay until the end, not you're not just supporting this uh this channel, you are one of us. Okay, you are one of those AI artisans that wants to build a a better world. So, watch the next video and u see you. Ciao.