Transcription
This video is a step-by-step tutorial to fine-tune this GPTOSS 20 billion on your own data set. I will be using my own data set which contains information about myself and we will fine-tune this open AI's newly released open weights model on this data set because this GPTOSS 20 billion doesn't know about myself. This model doesn't know who Fahadhat Miraa is, where does he live and all that stuff and that is what we are going to do.
So we are primarily going to fine-tune this model on this task which means we are going to do supervised finetuning. I will be using hugging face TRL library for post training or supervised finetuning and then we will also test the model if it is able to recognize about myself. If you are interested in learning more about these models, architecture, how to get them installed locally, just go to my channel, search with GBT OSS, and you should be able to find videos about both the variants 20 billion and 120 billion. If you're looking to fine-tune the 120 billion one, the instructions will be the same. So, let's get started.
First up, I'm going to show you my system configuration. I am using this Ubuntu system and my GPU card is Nvidia H100 with 80GB of VRAM. If you are looking to rent a GPU on very very affordable price, you can find the link to master compute in video description with a discount coupon code of 50% for range of GPUs. So please do check this out. I haven't bought it. I'm renting it because I don't know why I keep repeating this thing that I'm not buying this. I'm renting this. But still some people just say okay it is 80 GB VRAM Nvidia H100 we can't find it. You can it's spin it's fraction of cost. I think this whole video which I'm going to make maybe will cost you uh 70 cents or $1. That's it. So it's very cheap these days for one or two hours. That's all you need to do this thing including learning.
So let me create now virtual environment with GD and then we will proceed with the installation. So you see that it is installing all the prerequisites. Shouldn't take too long and that is all done. Next up let's install torch and then transformers plus which is parameter efficient finetuning and TRL from hugging face library. PEFT simply means is that what happens is in supervised finetuning that you use some technique called as Laura or Lorax adaptation you take a base model and then you create another set of layer with weights or small layer which you fine-tune as per your own task. So you don't touch the weights of the actual model. You just create a new layer on top of it which is called as Laura and that technique is parameter efficient finetuning and that is why we are installing this. I have done detailed videos on it in very simple language. So if you're interested just search my channel and you should be able to find those videos. Plus while you are at it please press that like button and subscribe to the channel as that really really helps a lot. please.
Okay. So, let's wait for it to install all of these things and then we will move forward. Meanwhile, let me also introduce you to the sponsors of the video who are Agent. Agent is the world's first multi- aent workforce desktop application empowering you to build, manage, and deploy a custom AI workforce that can turn your most complex workflows into automated tasks. So, please do check them out.
Okay, everything is installed. Now let's log into the hugging face because you might need to download or push your model or your data set to hugging face. Uh so that is why I am logging in because we will be writing to huggingface. So you would need to go to huggingface.co in your profile grab the right token not read and that's a free token which you can grab from your profile. Okay, I think I have I used the read token. That is not correct. I'm going to need the right token. So, let me enter my right token again. Give me a second while I grab that. I'm just doing it in real time so that uh you would be able to see how long does it take. So, pardon me. There you go. This is my right token. And that should show you HF. Right. That is the name of the token. You can give any name. Okay. So, that is good.
Now let me launch my Jupyter notebook and then we will do all the finetuning in my notebook environment. So let's wait. So my Jupyter notebook is launched as you can see. So let's begin the finetuning. For fine-tuning you need two things. Your data set. Secondly you would need a model. The model we will download later. First let's get the data set. I already have this data set in hugging face free Apache 2 license. Feel free to use it, modify it, do whatever you like. It's a very small data set. I just want to show you um a sample.
Now the thing which you need to understand here is that the data set format because that can make or break the training. So the GPTO SS models were trained on the harmony response format for defining conversation structure, generating reasoning output and structuring function calls. So the format the harmony one is designed to mimic the open air response API and this uh format which you can see on your screen is already optimized for that harmony one which you can readily use with open AI's model. So you see we have a system role which instruct u the model it's a instruction for the model as what model needs to do. Then there is a user one which which is a input to the model. Then we have assistant which is a response from the model and then you can either enable thinking or not. So let me click on this. So you see this is system. Then there is a content which actually is the input or user in this case. And then we have some uh user role content and thinking is null. You can also put in some thinking if you like. So some in some places I have put thick eggs in some places not just to give it a representative data and most of this data is uh fictitious. So don't really get too excited. Some of them is correct but some of them is not. Anyway so this is the data set as I said feel free to use it download it in order to get this data set. We already have logged in hugging face just run this. So this is going to download this small data set and it is going to print first record and there you go. So we have our data set and we have print printed first record from there.
Next up let's download the model. It's a simple uh command with the transformers. We are downloading the model and its tokenizer. And the model is being downloaded. And you can see that I am using um this eager because it just improves the performance of the model and that is why I'm simply using this eager in the attention. And then the rest of it is same as before where we are using this quantization and uh open uses something called as MXFP4 config for contisation and this is very specific to open a models that allows them to use mixed precision training with a special 4-bit floatingpoint format called as MXFP4 that is optimized for AI workloads for this model. So let's wait for it. And the model is now downloaded.
Let's first do a test of uh asking model who is Fad Miraa. And funnily enough, I haven't tested it yet. So it might know about me. That should be fun. But anyway, we'll see. So I'm going to run this now. And as it runs, you can see the VRAM consumption, which is uh around 45 gig of VRAM. And there you go. look at the response of the model in a typical chat GPT uh uh you know kind of fashion where some knowledge cut off date and all that stuff reasoning is medium I have discussed all of it in my previous video now you see it is talking about some path it doesn't know about it that it has got the name wrong then some journalist or musician and then searching lawyer and then Afghan Canadian academic so you see um it doesn't know about myself. So let's try to fine-tune the model on my data set so that it would know who the actual FAMA is. Let's do it.
Okay. Now for the fine-tuning, let's first get the PEP configuration. PFT, as I said earlier, it is parameter efficient finetuning. It is a technique that allows you to fine-tune LLMs very efficiently by updating on only a small subset of parameter which reduces computational cost at memory usage. And this is where you can use methods like Laura low rank adaptation which adds trainable low-rank matrices to existing weights without modifying the entire model. So let's run this and you can see that we are also updating all the attention layers plus also some of the projection layers because it's a mixture of expert parameter and then we are also setting the rank to eight. You can increase it but it is going to increase your VM consumption and it is giving you how many trainable parameters are there.
Next up let's set up all the training parameters. So if you look here primarily it is a TRL library which we are setting and we are importing the SFT config. Now I already have done video on it explaining all of these parameters but very very quickly we are setting up here learning rate which is a learning rate for AdamW optimizer. We are setting gradient checkpointing which primarily saves memory by recomputing activations during backward pass and then we are just setting epox to one which means that one full pass over the data set that's it we are setting up logging every step we are giving four samples per GPU per forward pass and then after every four steps we are accumulating the gradients plus we are setting the max sequence length for tokenizer or model input to this you can increase it it is going to increase the VRAMm but if you are have a lower VRAM maybe you can try setting this max length to lower but it is going to take more time and then for uh observability we are using this tool track which we already have covered on the channel so let's run this and that is done.
Next up let's do this uh training so you see all we are doing in this training we are giving it data set we are giving this uh model we are giving it the arguments which we have defined and that's about it and we are starting the training. So let's run this and there you go the training has started. So let's wait for it. So training is running the loss will keep coming down as it progresses as it learns more and more. Let's also check the VM consumption. So it is consuming now over 52 gig of VM as the training goes on and it will fluctuate around there I think but remains under 60 62 gig of VM. I'm not sure if it would be able to show me the track but let's try it out. Okay. So I okay sorry because I'm not in that um Honda environment. Let me quickly go there and try that command again. Let's wait for it. I just want to see there you go. So it is showing me the training at the moment. You see as the training progresses all of it which I which I discussed earlier around allocated memory memory usage it is showing you in the real time. You can keep looking at it and then you will see that memory usage is going. This is the training which is happening. What is the learning rate? Number of tokens you this is where it will show you the loss which will keep dropping as it goes. And this should be somewhat similar to this thing. You see the loss is coming down and this curve is also coming down. It it also fluctuates of course. Anyway, so I will wait for it to finish and then we will see what it has produced.
And there you go. It has um done the training. Took very very uh quickly because it's a very small data set as you can see. Now we already have pushed it to hugging face because we set it this push to hug to true and we have given this output directory. So it has already saved the new model with tensors which is a very precursor Laura tensors. It's a small one and you can merge it with the actual model too. But you see that if you don't want if you haven't done that you can manually also push it to hugging place or save on your local system with these two commands. But we already have that.
Now let me quickly show you this on my hugging face. If I go to my hugging face repo and if I scroll down there you go. You see this is a model which it has already uploaded after fine-tuning it. And these are the same files. You can see that this is a tensor file which is very small. You can also as I said you can merge it with actual model and then upload the whole model. And now let me show you how you can merge it all. So you just need to okay I'm just going to go to I'll just say maybe I'll just go to my notebook that will be easier to show. So this is what you need to show do where you will be just importing it getting the tokenizer getting the model load and everything and then simply you will be using this merge and unload method. That's it. So let's run this and the model is now merged. you can talk with it.
So let me ask it something like uh I'm just doing a basic persona identification. So I have given it the system instruction and then I'm asking the model who are you and what do you do? Because basically this is what we have done. We have fine-tuned this model on the task of it being Fad MSA. So let's try it out. And there you go. Even with that small data set, model has now clearly knows who Fahad Miza is that it's an AI YouTuber and then it knows all about myself from that data set and now it is not hallucinating. It is not making up anything. It knows where I live and everything. Very very nice. So you can just keep asking the questions. For example, uh you can ask it you know where do you live or something like that and it is going to answer that.
And as the model is fully loaded onto the GPU, let's see the memory consumption. So it is consuming over 48 gig of VRAM with that Lora. So for instance, if you ask it where do you live and then run this, it is going to run it. Let's check the VM consumption again. So it is consuming just over same thing 481. And there you go. So it is telling us you live in Sydney, Australia and love hands-on tech demos and all that stuff. So you know the better your data set is the more training you do it will be uh the model will be more performant.
Now it is all local, all free, uh all private. You can simply fine-tune it on any task. And similarly, if you want to find out finetune it on some preference data where you are very rigid about uh preferred or rejected answer, you can use direct preference optimization technique in order to find tune. It depends upon your data set. And as it is a reasoning model, you can also use any GRPU data set. But make sure that it is in the harmony format just like this one the data set which I showed you earlier. That's it. Please like the video and subscribe to the channel as that helps a lot. Thank you for all the support.