Transcription
Today, you'll learn how to fine-tune LLMs in Python for use in O Lama. I'll walk you through it step by step, give you all the code, and show you how to test it out. So, let's go ahead and get started.
Now, first, I want to discuss what fine-tuning is and when you should actually do it. So, fine-tuning is taking a pre-trained language model and teaching it to be better at your specific task. So, think of this like hiring an experienced chef and training them on your restaurant's particular recipes rather than teaching someone to cook completely from scratch.
So, here's how it works. Instead of training a model from zero, you start with something like GBT or Claude that already understands human language. Then you feed these examples of your specific use cases. So maybe customer service conversations, legal documents, or medical records. The model then adjusts its existing knowledge to excel at your specific domain using those examples.
Now, this is completely different from parameter tuning, which is just adjusting settings like temperature or top K to change how the model performs. Parameter tuning is like adjusting your car's radio, whereas fine-tuning is like teaching your car to drive in a completely different neighborhood.
Now, when should you fine-tune? Well, there's three main scenarios. First is when you need consistent formatting or style that prompting alone can't achieve. So something like you want to output a particular JSON model or something or you want the model to write in a very particular format. Second, you have a lot of domain specific data that the model just hasn't seen before. So things like advanced medical records or customer service logs or information that it just wouldn't naturally know because it's very specific to what it is that you're doing. And then third, when you need to reduce costs by using a smaller specialized model instead of a really massive larger one.
Now, the key advantage to fine-tuning rather than training from scratch is that you need way less data and compute power to do this. Instead of millions of examples and months of training, you might need thousands or even hundreds of examples and maybe minutes to train or hours to train, not weeks. But here's the catch. When you do fine-tune, you can make these models worse at general tasks, but you will make them better at yours. So, if you do fine-tune one of these models, keep in mind that it will get worse at general tasks, at least typically, but be much better at what you're fine-tuning it to do.
Anyways, with that information out of the way, I want to show you how to do this fine-tuning. So, let's go over to the computer and I'm going to show you how to set this up in Python with some example data that I have.
So, I'm on the computer now. And the first step that we need to complete if we want to fine-tune a model is we need to gather the data that we're going to fine-tune that model with. Now, this is the most important step. If you have bad data, you're going to have a poorly fine-tuned model. So, make sure that you take your time here and gather the correct data.
Now, for this video, I've just used AI to generate a pretty simple data set which demonstrates HTML extraction. So, you can see that I have some HTML like a div, an H2 tag, a price tag, whatever, etc. Okay, there's a bunch of information kind of inside of here. And what I'm expecting the model to give me is a nicely formatted output that tells me the name, price, category, and manufacturer of this particular data. Now, this is very specific because I just want to extract data from these types of tags. However, you could do anything as advanced as you want here. And you can see that I have 500 examples in this JSON file where I have some sample input and some sample output.
Now, this can be literally anything that you want because all you're doing to fine-tune the model is you're giving it example prompts and then example outputs of what the result or the answer from the LLM should be based on that prompt. So, you can use customer data again, medical reports, files, whatever. In this case, just simple HTML extraction. So, you can see kind of how it works and I can compare it easily in this video. If you want this specific data set, I'll leave a link to download it in the description. Okay.
Now, for this video, we're going to use something called Unsloth for doing our fine training. This is open source. It's free to use, and it's just extremely good and very fast at fine-tuning these LLM models. All right.
Now, what I'm going to do is I'm going to go over to this fine training notebook that I've already set up. For the purpose of this video, I'm not going to write all of the code from scratch. I'm just going to leave a link to this notebook in the description that you can download or that you can connect to in Google Collaboratory. And then you can use your own data and start fine-tuning your own models.
But before we get into that, here's a quick idea that's worth stealing. If you're already fine-tuning LLMs or building tools with Olama, chances are you're solving real problems for real users. Maybe it's a niche SAS, an AI agent, or a quick side hustle for earlystage businesses. Now, imagine giving those users a fully functional website, a storefront, booking system, portfolio, whatever it is, without ever leaving your app. No templates, no drag and drop builder to maintain just one API call. Well, that's what Tenweb's AI website builder API does. It's not a template engine or a static generator. It's a real API that spins up fully edible WordPress-based websites, structure, content, design, images, e-commerce, etc., all in under 60 seconds. With one API call, you can generate a complete website. Your users never leave your dashboard. You get full white label control, so your branding, your domain, your customer relationship. And Tenweb handles the hosting, WordPress management, and technical infrastructure while you focus on your core product. This is how Tenweb powers websites with over 2 million users. And now you can do the same thing without building anything. Whether your user is launching a restaurant, coaching business, or online store, the site comes complete with hosting, security, mobile optimization, and full e-commerce capabilities. All powered by WordPress, the platform that runs 62% of the web. Now, if you're ready to turn your next big idea into reality, then visit my link in the description and book a strategy call with the 10 web team to see how they can help transform your business.
Okay, so how does the fine-tuning work? Well, you can fine-tune this locally on your own computer by running all of the code that you see right here and that you can download from the link in the description. But unless you have a really powerful GPU, something like a 4080, 4090, something even more powerful than that, then this is going to take a very long time. So for most of you, I suggest that you use something called Google Collaboratory. This is a free online code editor environment provided by Google that allows you to connect and use super high-end GPUs. So you can do all of your training in Google Collaboratory. We can then take the model that we train here and we can download it to our own computer and run it in Olava, which is exactly what I'm going to show you how to do.
So, first step here is open up this notebook. Again, I'll leave a link in the description if you're doing this on Google Collab, and connect to a runtime. So, you should see a button here that says connect. We're going to press that, and we should connect to a T4 instance runtime, which is a T4 GPU. Now, if you're not sure, you can press on this right here, and it's going to show you what you have available. So, system RAM, GPU, RAM, and disk. And it will show you the runtime you're connected to. If that didn't work, you can go here, and you can go to change runtime type. And then you can make sure that you select T4 GPU and that you're running in Python 3. Okay.
So now that we're connected to this runtime, we can start running the various cells that I have inside of this notebook. Now, because we're going to train this on our own custom data, we need to bring in this file. So what I've done is I have this JSON file and I'm going to open it up here. But in order to open it in Google Collaboratory, I need to upload it to Google Collaboratory. So, I'm going to press on this file button here on the left hand side and I'm going to go to this little upload button where it says upload and I'm going to select this file. Again, you can download this from the link in the description. Okay. So, once this file is uploaded, you should see it here. And what I'm doing is I'm simply opening this file and loading it as a JSON file by providing the name JSON extraction data set 500. Now, you can use any type of data that you want. Again, just make sure you get it in this format where you have some input and then some output. Okay. And the output should actually be a string. I'm going to convert this to a string in 1 second because right now it's a JSON object. All right. So now that we have that, I'm going to run this. And you can see that I just printed out the first example. And I'm successfully loading in this file that I've uploaded. So I'll go ahead and close that window.
Next, we need to install the various dependencies. In my case, I'm in Google Collab. So I'm going to run exclamation mark pip install. For now, I'll remove the uninstall command. I just had that in case I had something installed beforehand. Okay. This is going to take probably a minute or two to download and install. Once it's done, I'll be right back and we'll continue.
All right, so that install has finished here and now we're going to move on to the next step here, which is related to the GPU check. However, as it says here, the following packages were previously imported in this runtime. You must restart the runtime in order to use the newly installed versions. So, I'm just going to press on the restart session button here just to get this reloaded. And once this restarting thing is done, okay, we're good to go. We can move on to the next cell.
Now, if you are running this locally, the install might look a little bit different and you are going to need to have CUDA installed on your system and be using an Nvidia GPU in order for this to work or at least for it to work quickly. That's why I like using Google Collab because everything is already set up for us. So, I'm going to go ahead and press run here. I'm just going to run this cell and check if I have a GPU available and if CUDA is installed. I should get two TRS here or sorry, a true and then a GPU value. So CUDA is available and I'm using the Tesla T4 GPU in this Google Collab instance or runtime which is exactly what I want.
Okay, so now we can move on and we can start actually setting up our fine-tuning training. Now here we need to pick the model that we want to fine-tune. Now I'm going to pick a really small model because I want to do this in a small amount of time. I don't want this to take days or weeks to train. And the model that I'm using here is the Fi3 mini model. Okay, you can look up this model if you want to see more information about it, but you can fine-tune any model that you want that's open source. So, for the model name here, you can put any Unsloth model. You can go to the Unsloth documentation to find all of the models that are available, but pretty much any open source model you can do here, like Llama 3.1, for example, Mistl, Mixel, etc. You can put them all here. Then, we're going to set the sequence length. Now, in this case, I'm just going with 48 tokens. This is the maximum number of tokens that the model can handle as input for the DT type or the data type. Don't worry about that. We'll leave it as none. And this just means that we're going to autodetect what the data type is.
Now, what we need to do here is just load the model that we're trying to fine-tune. So that's why I put the model name here. And I'm getting the model and the tokenizer from the fast language model from Unsloth. And then I'm loading this pre-trained model, which is fi3. Giving the maximum sequence length load in 4bit is equal to true because this is a 4bit model. Okay. So I'm going to go ahead and press on run. That's going to load the model for us. It will need to download it if it's not already here. So that can take a second. And you might see some stuff like this where Unsloth is patching our computer. Don't worry, that is totally normal. And once that is finished, I'll be right back and we'll move on to the next step.
All right, so the model is now downloaded here. Again, I picked a pretty small one just for this video. If you pick a larger one, this can take a much longer amount of time. And next, what we're going to do is we're just going to pre-process our data. So the day that I have, remember, comes in this format where we have some input and then we have some output. Now, what we need to do is we need to put this in just one single string that we can send to the model when we're fine-tuning it. So, what we're going to do is we're just going to use this format prompt function that I wrote here where we're going to have an input. That input is going to be equal to whatever the input is from our data set. So, you'll likely have to change this function to work with whatever type of data it is that you have. Then I have a new line character. Then I have the output. And what I've done here for the output is I've gone and I've grabbed this JSON object. I've converted it into text. So that's what JSON.dumps is doing. I'm taking a JSON object and converting this to a string. So I'm doing that for my output. Okay. And then I put this end of text tag here so that the model knows, okay, this is the end of the text. That's it. There's many different ways that you can format the prompt, but this is what I want the prompt to look like where we have this input and then this is the expected output. Okay. I then have my formatted data where I essentially just run through all of the items that I have in my file and call this format prompt function on them. and then I convert this into a data set.
Now, if you try to run this, you might get an error. That's just because this first line up here, we may need to rerun where we're loading in the file because we reset the runtime. So, if we scroll down here now after rerunning that, we can re-execute this cell and we should be good to go and we now generate this data set. That's just because we need this data set object when we're running this trainer. Okay.
So, now what we're going to do is we're going to use something called the Laura adapters. Now, I'm not going to go into too much depth on exactly what this is doing, but this line right here is essentially going to add the kind of layers that we need to our LLM in order to actually do this fine-tuning. So, again, this can get very complicated. You don't need to understand all of the parameters that are here. And if you do want to mess with them, you are going to have to look these up or maybe use an LLM to explain them to you because there's a lot going on and there's not enough time for me to explain this in this video if I want to make it short. So, in fact, if we want more information on this, so we're in Collab. I'll just highlight this, right click on it, press explain code. This is going to use Gemini to explain the code to us. And then we can actually just see what the explanation is. And that's going to be more accurate than me walking through everything. So it says the selected code applies the Laura low rank adaption method to the language model using this. Here's a breakdown of the parameters. And then it tells you exactly what these are. Okay. So use Gemini within here if you want to know about all the parameters. Either way, we're going to go ahead and run this. So, we're going to create this new model now by kind of adding these adapters to it, which then will actually allow us to perform the fine-tuning, which we'll do now. So, you can see it patched 32 layers with 32 QKV layers. Don't worry too much about that if it means nothing to you.
Okay, so now that we've loaded in the model, we're going to set up the trainer. Now, the trainer is actually the thing that's going to perform the fine-tuning for us. People much smarter than all of us have written all of this code. So, all we have to do is simply use it. So, again, you can read through all the parameters if you want, but what I'm using here is the SFT trainer. Importantly, I pass my model. I pass the tokenizer for the model. This is actually going to convert our string into the token so it can be understood by the model. I pass my data set which I created earlier. The uh field for my data set is text. That's important because if we go look at the data set, you can see that where is it here? We created the data set with this dictionary where it says text and then it has all of the values. Okay. And then we have a few other things in here that you can look at. For example, the maximum sequence length that needs to match with what we had before. And then all of these training arguments. Again, I'm just going to leave these kind of all default, but you can adjust them if you like. All right, so now we're going to run this. We're going to initialize the trainer. And now that that is created, what we're going to do is actually train the model.
Now, this step is going to take a different amount of time based on how many examples you have and the different settings that you configure here. The more examples you have, the longer this will take, but the better performance you will get. The larger the base model is that you're using then again the longer this will take. In my case I'm using a very small model with a very small amount of examples. So my performance won't be great but this should train in maybe a few minutes not hours or weeks. All right. So we are training now and you can watch this window and you'll kind of see the progress. So once this is finished I'll be right back and then I'll show you how we can actually test this model, make sure that it's working and then download it and use it with Olama.
All right. So the training step is finished here. Just as an FYI, that took about 10 minutes to run. So the next step is we're going to set up the model for inference just so we can test it and make sure it works before we download it. So in order to do that, pretty much all you have to do here is modify these messages to be the messages that you want to test. So you can see I have some RO, which is the user, and then I have content, which is the prompt I want to test. You can place multiple messages here, and you can see that this message is just similar to one of the examples where I'm doing this kind of HTML text extraction. You don't need to worry too much about these lines right here. We're just kind of putting the input and output into a format where we're able to actually send it to this model at least in the inference mode. And then we can go ahead and press on run. That will just take a second. It will run the inference. Then you can see that we get the output. So we have user, right, which said extract the information. And then if we go over here, you can see we have the assistant and then we get the output in the format that we specified. Okay. So we can see this is indeed working. And we could test this with multiple other messages to make sure it is working to our liking before we go ahead and download it.
Okay, so now that we've assumed, all right, this is good, it's working, we've tested a few times in Google Collab, what we want to do is download this model so we can actually load it into a llama and then start using it more permanently on our own computer or our local machine. Now, this step does take a while, so bear with me here. What we're going to do is run this where we say model.save pre-train gguf. GGUF is the model format that O Lama understands. So that's why we're downloading it in this format. So I'm going to go ahead and press run. And it's now going to essentially create this output file for us. Download it to Google Collab. And then after that, we need to manually download this ourself so we can save it to our computer. Now this can take a long time. It can take 10 minutes, 15 minutes, 20 minutes. So just bear with it. Be patient. And once it's done, you can run this cell. This cell will then download it to your own computer, then we're going to move on to the next step where I show you how to actually load this into O Lama on your local machine.
So once this is done executing and you've ran that final cell, you should have a file downloaded to your downloads folder that's named something like this. Unsloth.q4 km gguf. You're just looking for some file that ends in gguf. This is going to be a pretty large file and it will take a long time to download. In my case, both those steps took about 25 minutes to run. So, just keep that in mind. This will take a while, especially if you have a slow internet connection when you're downloading the file.
Okay, now that we have the file, what we need to do is add this to O Lama. So, we need to create something called a model file. So, what I'm going to do is I'm going to open up my terminal. I'm just going to do this all from terminal commands to make our life a little bit easier. And I'm going to open this up. Okay, I'm going to zoom in a little bit and we're just going to test and make sure that Olama is installed on the system. So run the lama command. Make sure you've got that downloaded and installed for the next steps to work. Okay. From here, I'm going to go to my downloads folder. So cd into downloads. And from the downloads folder, I know that I have this unsloth kind of Q4_K_M file. So what I'm going to do is I'm just going to make a new directory. So I'm going to say mkdur lama- test. Okay. If we go to downloads now, we can see this folder is created. I'm just going to drag this file into the folder here. So now it's inside of O Lama test. Okay, I'm now going to go inside of O Lama Test and I'm just doing this to kind of organize things, but you don't need to follow this whole setup here. And what we're going to do now is we're going to make something called a model file. Now a model file defines a custom configuration for a model that you want to run in O Lama. So we need to make one of those. Now to do that, we can make a new file. So we can say something like touch and then we're going to call this model file. Okay, just exactly like this with the capital M. That's going to make a new file for us. If we ls, we can see it here. Now to edit this, we can do nano model file like that. And then I'm going to paste in what the model file should look like. Okay. So it's going to say from do slash and then this needs to be the name of your file or the name of your model. So the local file that we had is called this. So I'm just going to copy the name by going to rename copy and then pasting it here. So we're going to say from/unsloth.q4_k_m.gguf. Again, this is kind of the base model. So we're saying from this file right here which is in our local directory where this model file exists. Then we can specify some different parameters. So we can do the top p the temperature the parameter stop. So user end of text because that's the way that our model works. And then we have a template. So the template is going to be user prompt and then assistant. And then we just have a system message. So if we want to tell the model to do something specific, we can. In this case, we just say you are a helpful AI assistant. So this is a very simple model file. This is how we're going to be able to load in this particular model and use it in a llama. So now we are going to save. So I'm going to hit Ctrl X and then press Y to save the model file. Go ahead and press enter. And now you can see if we open up the model file again. So nano model file. It looks like this. Again, we can get out of that.
So now we have a model file and we have the model downloaded. What we need to do is add this to a llama. So in order to add this to a llama, we're going to type lama and then create and then we're going to give the model a name. So I'm going to say this is an HTML model. Okay. And we're going to do -f and then model file. What this is going to do is add a new model configuration to lama for us using the model file. So let's go ahead and press enter. That's going to take a second. And now if we want to see if this is here, we can type lama list. And we should see that we get the HTML model that was just created.
So now if we want to run the model, we can just type lama run and then the name of this which is the HTML model. It's going to take a second to load up and then we can paste any message inside of here and we should see it working. So let's go into our data set. Let's just copy one of the ones that we have just to keep things simple. So let's copy this right here. Go back and paste it. Okay. And I just cleared the screen here so it's a bit easier to see. Okay. And then you can see after I put the prompt here and press enter, this is the result that we got. Now this isn't always going to work. So I can put this a few times and you'll see now look like it gives me a different output. That's because obviously we had a small data set. We're using you know a pretty poor model not one of the massive models but generally it is working in the way that we wanted it to work and more examples pass to this will give us a better output. Either way this is a now custom fine-tuned model loaded in a llama that we can run locally on our own computer. We could connect to it from Python or something and use it in our application. And that is going to wrap up this video. Now, I know this is not going to be perfect. You do need to experiment with this, modify some of the parameters, but I wanted to show you how to do fine-tuning, talk about what that actually is, and give you a step-by-step guide. So, hopefully this can get you up and running as fast as possible. All of the assets for this video will be linked in the description in case you need them. And with that said, I look forward to seeing you in another video. [Music]