Transcription
Hi, and hope you are doing well. I'm Jadi, and today I want to show you something very cool: a program to help you running a GPT thing, in more technical words, an LLM or even an SLM on your own mobile or tablet. This is a very nice project. It's called Pocket Pal, and it's started by Ascar Orbani, a great programmer, at least based on this project. The good thing about it is it started a few months ago. It was on my to-do list to be introduced, but it took a long time, a few months, and I was observing the project getting better and better technically every week. For example, the last commit to this project was only 3 days ago, and the project got very good traction on the internet. It has 384 forks and more than 4,000 stars. The reason that you don't see me starting it is because I'm not logged in. If you use the project, it's good to store it on GitHub. There were also, even I think, more than 10-15 contributors to the project.
The idea is kind of simple, but the execution is very nice. The idea is having an LLM running on your phone or on your tablet. As we may know already, LLMs are large language models. What is running behind things like GPT? You have an LLM here. Lots and lots and lots of processing made a huge file, a large language model, tens of gigabytes. And you have a program which can speak with it. So when you have the model, you need a program to let you type something here, send it to the LLM with a protocol, and then get back the answer and show it here. This is ChatGPT. So this is the uh distinction between the model and the program. You have GPT here, it's a model, and you have ChatGPT using that specific model. So it can be different things. And nowadays, we have lots of open-source models like Llama, like Gemma, like Deepseek, and many others.
Okay, our phones do not have enough capacity and processing power to use this one. So they also created some called SLMs, small language models. Still, we are speaking about 2 GB of a model size. So it's not that smart. What this project does is using lots of nice open-source tools. There is a list here, and it's very clear. So the idea is having an LLM on your machine. Now we know we call it SLM because it's small, but the technology is the same. It's just smaller. And using llama CPP to talk with it and creating a React Native project, React Native project here. So you have a text box, you have a text box here, you write whatever you want, using llama CPP, it goes to the SLM, returns back, and shows the results here. This was the first step when I saw the project.
Now there is another cool feature. You can go through the configurations here, choose one of the already available models, download, and load it. Very, very straightforward. Also, if you want, you can connect to Hugging Face, use your models there, or copy your model on the phone and just use it directly. Everything is possible. Uh, let me first also, another great thing was it's very, very actively developed and used by many people. So there is an iOS and Android already available on the market. Pocket Pal AI, just search for it and install it. Also, if you are on an Apple machine, you can go with Pocket Pal AI. Let me show it here so we can both see a few features.
Okay, here you can go to the Play Store, search for Pocket Pal, and install it. The original APK is small. It's no more than 20-something megabytes. But then you need to download one of the models. Now I have it. Let's run it. Nothing there. So I will go with download available models. You have many different models. So I will download Gemma. Uh, it's 2 GB. Will take a few minutes to download. Let's wait.
Okay, while I'm waiting for the download to finish, let me introduce you to a couple more features which I really liked. One is you are able to configure your model after downloading your model, uh, fine-tuning it, just playing with it. So you will have a better understanding of the whole LLM thing. Also, you have a concept of PALS, which is very nice. You can create an assistant pal based on one specific model and add a specific prompt. So it will work as a helpful assistant, as a criticizer, or whatever. And also you can have a role-playing part which helps you to have fun, have talk, practice your English, or whatever you want because you can, for example, say your name is blah blah, you are a mermaid in a sea, and I'm a captain, and we are searching for a treasure in a horror fantasy world, and then start talking to it, and it makes using your phone much fun.
And talking about phones, I want to talk about another application I use a lot. I've been using this app called Brilliant, and it's become part of my daily routine. Every day I spend a few minutes on it, learning something new, mostly math, but also logic, AI, and digital circuits. And soon I'm diving into regression and classification. What I love is how interactive it is. You are not just watching videos. You are actually solving problems as you go. It really helps you think better and understand the why behind things. Brilliant has thousands of lessons, and they are all visual and hands-on. It feels more like playing with ideas than studying, and it works. I've built up a 50-day streak on my phone, on my laptop. Whenever I get a free moment, I just open Brilliant and learn a little bit more. To try everything Brilliant has to offer free for 30 days, go to brilliant.org/judi, scan the QR code on the screen, or just click the link in the description. You'll also get 20% off if you go for the yearly plan.
Okay, our download is done. Now it's enough to load it. When loading it, you are loading a huge uh LLM or SLM into your memory. Another good point about this application is it removes it from memory to preserve your your memory usage when you are not using the app. And most simple, basic one, you have this "Hello, who are you?" what every LLM should answer. "Hi there, I am Gemma." Nice. What I've told you was you can go here. You can go to the Pals. You can add a new roleplay one here and say you are Sara. Your default model is this one. What the world? It's a pirate fantasy. Our location is ocean. AI's role is a mermaid, and user role is a pirate. Situation is exploring for treasure. Terrors. No treasure. And tone style is horror fantasy. You can ask AI to create a new prompt for this specific genre. You know most of the things which are happening on the LLM world, all are creating prompts. It's enough to tell this SLM or LLM when using it. Imagine you are in a fantasy world. You are you are a mermaid. I'm a captain. You are in a higher world, and this is the setting, blah blah, based on that, give me a prompt, give me an answer for this specific question. So now it's generating this huge prompt for this. Let's wait and see.
Okay, now that the prompt is generated, I can save, go to my chat, and now what I was doing, I started a conversation. I can change to one of my pals and say, "Okay, I want to speak with Sorrow in a roleplaying world," and say, "Wow, do you see that ship?" And wait for the answer from my mermaid companion when I'm pirating in seven seas. Anyway, it was fun. I enjoyed it. And sometimes I use it as a role-playing game just to fight boredom. And also, it's good to have an SLM running in your pocket if needed, without internet, without paying any subscription, and also enjoying as a programmer to see what is going on here or do a contribution. There are issues. One of the issues I would solve is when changing different pals and chats, I prefer a fresh start. This is my preference. Maybe others don't like it. Anyway, hope you enjoyed it. If you use the project, give it a star and a shout out for Ascar Orbani for the great project. Thanks.