Transcription
Hello everyone. This is Professor Pattern, and in this video, we're going to be looking at the knowledge-based section of our Open Web UI workspace, along with the document settings for our admin panel.
Now, let's start with a very simple use case. Imagine that you are a recruiter for a company, and you put out a job, and you had a bunch of different candidates apply for it, right? You downloaded all of their résumés, some that are in PDF format, some that are in Word format. Now, what you want to do is find out who is going to be the right candidate for the role. What you can do is create a knowledge base. So, the way that you would do that is from your home screen, click on Workspace, and then select Knowledge. Here, I'm going to select this plus icon, and you can fill it in with "Candidate Résumé Pool." And this is a collection of résumés that I have collected from some for some candidates.
Now, let's create the knowledge base, and then I'm going to add this content by uploading a directory. So, I'm simply going to select the directory where I've downloaded all of the résumés. Once I do that, you can see all of them have successfully loaded in.
Now, suppose that I want to find a candidate for a particular role. Let's go to our home window here. I'm going to take a prompt, and the prompt is "the right candidate." And basically, it says, "Based on the job description provided, take a look at the knowledge base and tell me who would be the best candidates for the job." So, the first thing I need to do is provide it the knowledge base, so "Candidate Résumé Pool." So now it knows it needs to look at this knowledge collection. And then, let's bring up a random job description. So, I found one for a Data Scientist from OpenAI's website. So, I'm going to simply select all and then copy this, and then I'm going to paste it here on the Open Web UI window. Let's run this, and we can see here that this is a Data Scientist role, and it requires someone with 10 years of experience, and some experience with Python and R, some ETL workflows, Tableau, Looker. So, here we can see that based on this job description, as well as requirements, it selected Alexander Monroe and Jessica Lawson as the two candidates that would be the best fit for this role. Uh, it said that both candidates have strong qualifications, but based on that, let's see if it said, "Okay, there we go. Alexander Monroe slightly edges out as the optimal candidate." So, we can even take a look at Alexander Monroe's résumé. So, this is what it looks like, and we can see that they are a Data Scientist with 10 years of hands-on experience. So, that's perfect.
Let's also try this with one more job description. So, this is a job for a Recruiting Manager. So, again, I'm going to simply copy this, and then let's go back here and say, "Find me the best candidates for this job." Okay, so here it said that based on the Recruiting Manager requirements, Peter Thomas does seem to have, um, a good amount of experience. It says, "They have 18 years of experience in senior recruitment roles." And let's confirm that by looking at at Peter's résumé. So, Peter Thomas, I think that's the one. And if I open this up, we can see that this is a person that has had a lot of experience with recruiting and seems like they are going to be the right candidate for the role.
Now, this works great, but how does it actually work behind the scenes? Well, to do that, let's go over to the Admin panel, select Settings, and then select Documents.
Now, the first thing that stands out is something that's known as the embedding model. Now, I have another video that documents exactly how AI models actually understand from human text, but if you don't want to become an expert on natural language processing, then you don't need to watch that video. If you do, I've linked it in the description. All you need to know is that the goal of an embedding model is to take the text that we provide and convert it into a numerical representation. The way that embedding models are trained is by providing it a sentence. So, for example, you can say something like, "Mitochondria is the powerhouse of the cell." Now, when you provide the sentence, the goal of the embedding model is to look for each word and all the words that are similar to that word. So, when it sees "cell," it now understands that "powerhouse" and "mitochondria" are going to be words that are usually mentioned in this context of "cell." Now, just like how we provided this sentence, we would give it a huge collection of documents, maybe the entirety of Wikipedia, is looking at every single word and all of the context that that word is mentioned in.
Now, at the end, you get something like this. It looks scary at first, but this is just an embedding, uh, projection. So, if you click on any of these words, like for example, "systems," we can see all of the words that are similar to "systems." And all of these, when I say similar, what I basically mean is that they are all numerically close to each other to the word "systems." If I say, for example, "Karachi," we can see that words that are similar to "Karachi" are Sudan, Cyprus, Algeria. Uh, if you take the word "constitution," words that are similar to it are Parliament, U, presidency, territory. So, all of these words are close to this particular word. And when I say close, I just mean like in terms of the overall distance between each point and that one word. So, that is the goal of the embedding model.
So, let's go back here, and we can see that the embedding model that we selected is the Sentence Transformer MiniLM L6. And we can find all of these different embedding models by going over to espert.net. So, on espert, we can see that there are a bunch of different pre-trained models. Now, each model is trained a little bit differently, and so far, we can see that the MiniLM is a model that's really fast, and the size of this model is going to be pretty small compared to all the remaining models.
Now, suppose that you want to try out a different model. Well, all you will do is just go over to that model page. So, suppose that you want to try this one, "all-MiniLM-L12-v2." If you select this link, uh, we can go here, and then we can simply copy this. Now, once we copied this, we can go back here to Open Web UI and then simply paste that. It's the "sentence-transformers/all-MiniLM-L12-v2." And then, once you do that, you can simply download this, and then you can start using this model.
Now, at this point, you're probably wondering, well, what's the difference between all of these different embedding models? Well, they're all trained slightly differently, right? They have their own sort of flavors. Maybe some are going to be trained on large amounts of text, maybe some are trained at the sentence level, maybe some have very specific vocabulary terms based on a domain, for example, healthcare or aerodynamics or finance. Now, you can search here on espert.net. You can see that there is the Massive Textual Embeddings Benchmark leaderboard. So, this is what we call the MTEB leaderboard set, and it tracks the performance of all of these different models across a bunch of different types of tasks.
Another way in which you can also search for these models is just click here on Spaces, and then select Models. Now, here, select Libraries, and then search for "sentence transformers." Now, if we sort by maybe most downloads, we can see that "sentence-transformers/all-MiniLM-L6-v2," that's the one that's by default available on Open Web UI. Now, if you want to try a different model, uh, we can see that we can maybe try the multilingual MiniLM model, for example, if you have different languages that we are working with. If we have maybe medical-specific vocabulary, we have this one. It's the "newml-pubmed-based-embedding," so PubMed is just a place where we can search for medical terminology. So, we have a lot of medical or healthcare-based papers. And if you scroll down, we can see the overall performance of this model compared to the "all-MiniLM-L6." So, this one does perform a little bit better for medical-related tasks. Um, if we go back here, we can see that we also have something like "nomic-embed" and we also have "gen-embeddings." Now, these are going to be in terms of licensing a lot more permissible. So, a lot of people like to use these models as well.
Suppose that this is the model that I want to try out. I'm simply going to copy this, go back here to my Admin panel, and then paste it. Let's select Download. And it's going to say that it's going to reset the knowledge base based on the warning that it says, "If you update or change your embedding model, you'll have to reimport all of your documents." And that's fine because we are simply uploading a folder. So, it said, "Embedding model is now changed to nomic-embed-text-v1.5." And if I go back over to my Workspace and Knowledge, we can see here that all of these résumés still exist. So, all we would need to do is just simply select "Add Content," "Upload Directory," and then select the folder, and that way it will get updated.
Now, let's go back here to the Admin panel, Settings, and Documents. So, what's actually happening under the hood? Well, the goal of the embedding model is to take all of our texts and convert them into a numerical format. Now, when we ask a question, say something like, "Find me the best candidate with SQL and Tableau experience." Well, first, what we'll do is convert that query into an embedding representation, and then what we want to do is simply find the most relevant pieces of text related to that question. So, similar to what we had here, um, all we will do is just find the, instead of words, we're looking for sentences or chunks. So, our goal is to essentially provide the large language model the query along with the five most relevant chunks. Now, we can select the most, the amount of chunks based on this "top_k" parameter. So, if you wanted to say like, "Okay, give me the five closest sentences or the 10 closest chunks of text," uh, based on this question, then that's what the "top_k" parameter is for.
Now, if you scroll down, you can see the size of each chunk. So, it says that the size is a thousand. Now, this again depends on whatever embedding model you use, whether it's looking at a thousand characters, words, or sentences. And then the chunk overlap is how much overlap there is between each chunk such that it retains some of the context.
Now, if you go here, the magic actually happens in the RAG template, right? We specify that the task is it needs to respond to the query. We gave it some guidelines along with some examples of what a citation looks like. Now, the output is "Provide a clear and direct response and use this based on the context." Now, the context that we provide to it are going to be the three most relevant chunks based on the text, and the query is whatever the user query was.
Now, suppose that you wanted to change some of these parameters. Suppose that you wanted to have more context, right? You want to also understand it from the LLM perspective. Maybe some models are going to have a huge context window, something like Claude 3.5 Sonnet, that has a context window of 200,000 words or tokens. So, you want to be careful. If you select a larger context size, then maybe you might end up filling up all of that memory.
Now, here, if you scroll up, we can also see that there's something else that's known as hybrid search. So, what happens if we enable this? Now, if we previously were looking at the just the embedding model, what we were doing was simply looking at the closest pieces of sentences or chunks related to the query based on where they lie in this embedding representation. However, with hybrid search, we also have two different parts. We have the embedding model, but we also have something that's known as the reranking model.
Now, all this is is basically another way in which we can find the most relevant chunk based on our query. So, here, for example, we can set for a reranking model as a default one, or if you wanted to, we can go back here to the MTEB leaderboard set and then search for specifically reranking. So, it would be right here, reranking, and then we can search for one, maybe like that's the top of the leaderboard. So, let's go back here. When we enable hybrid search, what it's going to do is that it's going to find the top three most relevant chunks based on the embedding model, and then the top three most relevant chunks based on the reranking model, and then based on these two searches, it's going to give us the three most relevant pieces of text. So, it's just another way in which we can really improve our process of getting the most relevant pieces of text that our model can compare with.
So, that is going to be the overall idea behind the document section of the Open Web UI interface. Hopefully, it gave you a little bit of an understanding about how all these components are kind of related to each other. If you wanted me to do a deep dive into any of these components or wanted to understand some things in more detail, let me know in the comments, and I'll definitely make another video on that. Thank you all for tuning in. I'll see you in the next one. Have a good one.