Transcription
[Music] thing first, it makes the management of documents easy. Go into that collection and take a look. Um, there's a section called pre-trained models, where it lists all the embedding models suitable for being used with Open Web UI. And here we go. I click open, and it's going to upload and compute vector embedding and put it in a vector embedding database for me.
Don't forget to give us likes, leave your comments, share us to your friends, and also subscribe to our channels. We have YouTube, LinkedIn, TikTok, and Facebook. That's about community. Don't forget that you have community so that you can get some models, uh, that people have done before.
On the second topic is the knowledge. This is related to RAG, right? You want to find an answer within your knowledge base or document sets or something like that. So, a new update is a very good thing. First, it makes the management of documents easy. So, yeah, I can go into that collection and take a look at the information or data or text inside it, right? That's pretty cool. I don't remember it having this feature back then.
So, if you are going to import or create a new knowledge base, you click this button, and you give it a name. NBA knowledge. Describe this. This is a knowledge base about NBA course, uh, will be helpful in answering business-related questions. Okay. And then you create the knowledge. Now, you have options to upload a file, upload a whole directory, sync the directory, and add text, like manually adding text content. So, I am going to upload a whole directory, and I am navigating to my NBA sources here. And here we go. I click open, and it's going to upload and index that, um, for me. Index, compute to vector embedding, and put it in a vector embedding database for me. And this is going to take a while because there are probably 10, almost 20 files inside that folder and subfolders.
So, while it's completing this, um, embedding creation, so let's take a look into how can we configure embedding models. See, there are 33 files. It's being calculated. So, the embed that is responsible for digesting, um, input files is located in the admin page, where you can access the admin page, admin panel, right here, and come to settings, and again, come to documents. So, in this, um, page, it allows you to specify the embedding models and whether you want a hybrid search or not. So, the default is a sentence transformer, and this is a field where you can specify what, um, embedding models that you want to use, but it has to be HES-based models. So, HES-based models, where do I know it is? So, usually, it's going to be relying on the sentence transformers, uh, on GitHub, and, uh, esper.net as well.
So, if you come to this website, um, there's a section called pre-trained models, where it lists all the embedding models are suitable for being used with Open Web UI, and you can compare performance on 14 datasets or semantic search and the speed as well. So, typically, if I remember correctly, a default go-to, uh, when you just pull, um, Open Web UI to your laptop, and, um, run it, it's going to be this guy, um, because it's very fast, speed is high, um, with minimal or very low impact on its, um, performance. So, yeah, this is typically what you see when you use, um, Open Web UI off the batch.
So, one thing you can change it by, let's say that you want to change it to this guy, right? You can go to the Hugging Face, um, model card, and you copy this one, and you paste it into this box. You paste it into this box. That's what I did. But be warned, if you change the embedding models, you need to recompute your documents. The reason is that different embedding models will result in different, um, vectors. Embedding, even though it's on it, computes the same document. So, when you change the embedding model, you need to redo all your knowledge here, right? It's going to be tedious for sure, but that's the thing, that's what you need to do.
Somehow, it's very slow. I think this is because, because, because, where is it? It's because I am using this guy, all-MiniLM-L6-v2 here we go, all-MiniLM-L6-v2. Um, it's a large model, larger model compared to, um, what you have, uh, the default value here. So, yeah, it's very slow. So, let me just show you that I want to kill the process here. I can do delete. So, let me just put it back to a faster model. But just for illustration purposes, I am going to use this guy. So, I will go to the Hugging Face and copy this. Go to my settings, documents, and paste this guy in and hit enter. If this works, yeah, it works. So, now the model has been loaded, downloaded to, um, the Open Web UI Docker or space.
Now, what you need to do is go back to whatever it is, and you see directory. This will reset the knowledge base and sync all files. Sure, let's do it. Oops, what is happening here? Seems like it's still, uh, okay. I think it's broken. Go to knowledge AI for business. Let's add NBA knowledge. Knowledge base about business and NBA. That's a nice. And then I will upload a directory. And for the sake of, um, demonstration, I will just use a smaller folder right here, and you can see that the embedding gets much faster. Okay, let's wait when it's done. Oh, what is happening? Uh, I think I killed the process in the mean, uh, while it was calculating. That's why it's getting slow. I think is there anything here? No. Okay. Yeah. So, I am going to upload the, um, consulting paper again.
So, while it's doing its thing, let's, um, dig a little bit deeper into the embedding models. So, here we go. So, this is the compilation of, um, sentence transformer models used it for creating vector embeddings as well as semantic search, and that is a leaderboard in Hugging Face Hub. So, MTEB is a dataset, dataset that is used to test, um, the model's performance. It stands for Massive Text Embedding Benchmark, and this is the leaderboard, and you can see here that, um, there are many, many models on this board. So, you can focus on open-source models, uncheck the proprietary ones, and you probably unselect something else. Let's see. I will just unselect everything except the sentence transformer. I want a small model as well. And this is the result of the filtering.
So, if you want to take a look, actually, if you want to use any one of these guys, um, what you can do is select one of them and do the same thing, copy this and put it in the Open Web UI box over here. Now, um, if you enable the hybrid, you have access to a ranking model. Why do we need to re-rank? So, when you do, um, RAG, the system will retrieve an X number of chunks that is the closest to, um, the query, right? The most related, um, chunks to the query, but you will get a better performance when you also re-rank those, um, selected chunks as well. And there are many, um, re-ranking models out there. So, you come here again to the MTEB leaderboard and you select, uh, re-ranking here, and this is going to be a list of, um, models that you want to use, right? You can select, um, anything like BGE-base-English, or if you try Sentence-T5-base, you can do that too. But in my, um, Open Web UI right now, I'm using this guy. It's quite big, I think. Anyway, so that's how, um, you configure your embedding model and ranking model.
If you're using a default sentence transformer, another possibility is to set up with, um, OpenAI, but you need to have that Ranger, um, model downloaded or, you know, use an API call to OpenAI, which will be like calling this endpoint, and also your key, OpenAI API key. You can select the embedding model here. I think there are three variations: small, medium, large for the embedding, and this is how many sentences you are going to compute the embedding per compute, right? For now, I am going to use the sentence transformer default on my laptop here. Did it change back? Yeah, I think it changed back automatically somehow. If I'm happy with this, I will save it right now. When I saved it, I can, let's take a look at this guy. I think it's stuck calculating or uploading stuff. I don't know. I think I have corrupted it somehow. So, let me just delete. Delete. Confirm. Let's, uh, just get started from nothing. So, make sure that this guy is okay. So, save and go to knowledge AI again. Okay, let's create knowledge and upload a whole directory. Um, let's go to NBA files and marketing. Select it. Okay, seems like I've done this so many times and it doesn't really go anywhere. Okay. Um, so, if it works, you can equip it to a model. One process, one document has been done. Let's try to use the knowledge.
If you want to access the knowledge base collection, what you need to do is put a slash there, and you will get to select what knowledge base collection that you want to use. For the LM to base, it's, uh, answer about. So, uh, here we go. Um, what is happening? Failed to add files. Could you find what you're looking for? That is really strange. What is marketing collateral? Is it going to work? No. Actually, here the context provided doesn't specifically define marketing collateral based on my knowledge. Marketing collateral refers to a collection of media. So, it's answering based on its, um, its knowledge because it said that it doesn't, uh, it couldn't find anything related to marketing collateral. And maybe because the document hasn't been uploaded properly yet. But again, if you can take a look at this, um, the citation, you can also see the relevant score, um, of this chunk, and this chunk appears in page number two of this file, right? Here we go. So, even though it doesn't really work perfectly because my knowledge base is somehow corrupted, it's very, very slow today. I don't know why.
So, let me just try to let it regenerate the answer again. Converting this to, um, here we go. Collateral refers to a collection of media and materials used to support the sale as marketing of a product or service. Is it in one of this guy? Let's take a look. Funnels. I'm not confident at all with this. Oh, what did I do? This is the first document that it thinks that it's related to the question. I don't see any marketing collateral here. Yeah, no. I think it hasn't been, um, the knowledge base hasn't been really updated to solve this question, right? Let me try it again. Share. What is marketing collateral? Collateral searching stuff. You don't need to, uh, enable anything here before. So, marketing collateral refers to a collection of. I think this is it. This is the actual answer. All right.
So, yeah, that's how it works, guys. Um, the knowledge base has been upgraded. You can easily come and delete the document you don't want. You can actually select that piece of file and see what's inside. Marketing one. Here we go. Yes, it only extracts, um, text, right? No images at all. So, be aware of that. If your document is, um, image or non-text heavy, I don't think you are going to get a good knowledge base search. Okay. Yeah, that's about, um, the knowledge. And I told you about embeddings too, right? Be careful that you need to choose the embedder type, and somehow, if you pick the wrong one, when you put it in this box, and you hit enter, this guy is going to bounce back very quickly, and that is going to be an alert message that the error has occurred.