Transcription
Hello again, my friends! This is Leno Tadros, and in this third and final part of the SQL LLM video, I would like to show you how to do the RAG operations with Llama Index. So, let's go ahead and get started.
All right, the first thing we're going to do is create a brand new file. We'll call it `02_llama_index_SQL.py`. You can call it whatever you want; that’s fine.
In here, I'm going to be bringing in some imports. I'm going to import the OS module, and you'll see why I'm doing that. I'm going to bring in `os.environ` so I can save the key for the OpenAI API key. Then, I'm going to use SQLAlchemy. In SQLAlchemy, I'm going to import `create_engine`, `MetaData`, `Table`, `Column`, `String`, `Integer`, and `Select`. These are things that have nothing really to do yet with Llama Index, but SQLAlchemy will help me a lot to dissect my URI for the database and understand how to create an engine out of it, metadata for the schema, and also all the tables and columns.
After that, I'm going to start using Llama Index from the core. I'm going to use a SQL database like we did with LangChain earlier. It's similar but more powerful when it comes to RAG and the Llama Index LLM.
We're going to import OpenAI. Remember, OpenAI is not the only way to do this. You can actually bring in this using Claude from Anthropic, you can use Gro, you can use Gemini, Llama, whatever you'd like. You can definitely try it with different things. Finally, I'm going to import OpenAI itself.
Sounds good? The next two lines are there because there are so many different ways to do this. I just wanted to show you different ways. I can say `os.environ['OPENAI_API_KEY']` and pass in my OpenAI key from my platform, open.com. Then, I'm going to actually open up the OpenAI API key directly coming in from the environment variable `OPENAI_API_KEY`.
There are a lot of different ways you can have an `.env` file for the environment, and it will load it automatically. You can actually pass the key itself in the call to OpenAI. There are many different ways to do that.
All right, the next part is something we've seen in the previous video with LangChain. I'm just going to create my URI. I'm going to let it know that I'm going after my localhost for a SQL Express instance called SQL 2022. The database name itself is called `ttb_DB`, and the driver I'm going to use is the `do_net_base_SQL_client_11` to bring it in.
Now, there are three very important lines of code that I would like to show you. The first one is that I'm going to create a variable called `engine`, and I'm going to use `create_engine` from SQLAlchemy to pass it that URI. That by itself will automatically create the engine ready for me to be passed to whatever I need.
There is another one here called `metadata`, which is also coming from SQLAlchemy. It says `create_all`, and I'll pass the engine. Maybe you do not want all 256 different tables, so maybe with the metadata, you get to choose which table you want to bring in to the LLM. For instance, later on, when you say `create_all` and pass the engine, that means I want the kitchen sink.
I actually do not want to learn about the relationships and the referential integrity between the tables and Sitefinity or whatever CMS system or SQL database. I'm just going to be passing the entire engine that I brought in, and I'll bring in everything—the entire schema from there as well.
Finally, I'm going to use the LLM—I'm sorry, the Llama Index core SQL database—to pass in the SQL database constructor. Here, I'm passing the engine that I brought in and the metadata object that I created based on all the schema coming in from all the tables.
Honestly, before I even get to any LLM work, I just want to see what this SQLAlchemy thing is doing for me. So let me go ahead and say `from SQLAlchemy import text`. I'm going to do a `with` statement here. I'm going to connect the engine, which is my URI for the database itself.
All right, I'm going to give it a name as a connection, and we'll say `row`. Then, we'll say `execute` the connection. I'm going to be passing `SELECT username, email FROM SF_user`. Of course, I don't want to know what the table name is or what the column names are. I just want to test SQLAlchemy itself. No LLM is involved here.
I'm going to say `fetch_all`, and now, if you remember from the previous video, that table has four different records in it: Leno, Kobe, Justin, and TTB. So when I actually print out each and every single one of these rows, I should see all of them.
Before I even go any further, I just want to make sure that this will work just fine. So let me come back in here. We'll say `python sl02.py`. There you go, this is the one. I'm going to run this, and if I've done my job right, I should be able to come back in here now and see all four records printed one per line using this `engine.connect` and iterating through all the different results coming in from the select statement of the fetch.
Indeed, it did! Notice here it brought in Leno, Kobe, Justin, and TTB, so it did the job correctly. But again, there is no LLM here; this is just SQLAlchemy. I'm fetching everything myself using a select statement. I just wanted to show you what SQLAlchemy is bringing to the table at this point.
All right, now the fun part starts! The fun part starts if I start bringing in some LLM. So let me bring in LLM here, and I'm going to say `OpenAI`. I'm going to set `temperature = 0.1` to keep it real—don't hallucinate on me! Then, I'm going to use the model `GPT-4`. Notice I'm not passing the API key, and the reason for that is I have an OpenAI API key that I got from `os.environ` automatically.
Again, I can delete these two lines and pass the API key myself if I want to. I just wanted to show you different ways than in the previous video as well. But here, I'm going to initialize my LLM to know exactly what temperature, what model, and there are other parameters you can pass to get the top K and everything else if you're familiar with how LLMs work.
All right, now let's go ahead and have some fun! I'm going to do this three different times to show you what exactly is going on. So this one, I'm going to be bringing in from the Llama Index core query engine. I'm going to import something called `NL_SQL_Table_Query_Engine`. This is a very important line. I'm going to create a variable I call `query_engine`, and I'm going to initialize this `NL_SQL_Table_Query_Engine`.
I'm going to be passing the SQL database object that I brought in earlier. This is the one that has the engine and the entire schema. I don't have to pass the table; I can delete this part completely and just say `llm = llm`. That will also work, but that means I'm bringing in all the tables in the schema because I said `create_all`.
All righty, but I have a choice here. I know that the agent I'm creating is about users, their roles, their profiles, and the link between them and the relationships. I'm not trying to get into the blogs or the news, so I can help the system out by saving money on the tokens, to be honest with you, and say `tables = [...]` and then create an array and pass the tables that I know that I need.
I'm not going to need anything other than these four tables, or you can just completely remove this and just pass the SQL database and the LLM and let the system figure it out. But there are too many tokens being used at that point.
Okay, so with the query string, I can actually say, "What roles does the user Leno at the training boss have?" If I go back to the database right now, I'm not going to be able to get this information from `SF_users`. `SF_users` will only tell me that there is somebody called Leno at the training boss.com. But I have to go to `SF_roles`, and I will find out that there are nine or ten different roles. One of them is administrator, another one is author, another one is designer, and there are a lot of different roles inside Infinity.
Then, there is the user profile that contains a lot of information about this Leno guy, right? Finally, I'll have something called `SF_user_link`. This table will actually have the ID—not the name, not the email address—but the GUID for Leno at the training boss.com, and it is associated with the GUID of the `SF_role`. This means this Leno guy, using an ID, could be an administrator, an author, a designer, and a backend user, so it might be repeated multiple times instead of this `SF_user_link`.
But I'm not going to explain any of that. I'm just going to say in English, "What roles does the user Leno at the training boss.com have?" The response here will just go ahead and use the query engine, which is the `NL_SQL_Table_Query_Engine`, passing that query, and I'm going to be printing out the response.
Would anyone like to guess if the system would be able to figure it out? Remember, there are 256 tables. I am helping it out by saying focus on these four, but even if I remove that, it will still be able to take care of it. It might take a little bit more time because there will be negotiation going back and forth between Llama Index and the LLM itself.
All right, let's go ahead and save this and run it. I'm going to go ahead and press `Ctrl + S`. Let me make this a little bit bigger, and we'll say `CLS` to clear. I'm going to say `python sl02.py`.
Drumroll, please! It will be really cool if the system finds out between four different tables what roles Leno at the training boss.com has. You can ask this question in so many different ways in English, and it will still hopefully be able to figure it out.
Let's give it a few seconds, and we'll see when the answer comes back. First of all, that's the first part. I didn't delete it, so it still tells me what the users are, and now it's doing the second part.
The user with the email Leno at the training boss.com has the following roles: administrator and backend user. This is amazing! This is correct; I am an administrator and a backend user as well. So it was able to figure it out and get the information from all four tables at the same time.
All right, folks, I want to go a little bit further than that. I don't want to do this operation every time. That's an expensive operation to get the schema for all of these things and come back. Is there a way for me, instead of doing this on the fly every single time and spending money for the tokenization, to take the entire 256 tables and embed them and vectorize them inside of a database like Pinecone AI?
I mean, you name it; there are tons of different vector databases out there that you can use. Even on your own machine, you can use Chroma DB to do that.
So look at how cool this is! If I come back in here and do it in a different way, let me go ahead and show you what this is. I'm going to actually bring in Llama Index core retrievers. So now I'm not using the table query; I'm using a retriever called the `Natural_Language_SQL_Retriever`.
I'm going to create my own variable and initialize this `NL_SQL_Retriever`, still passing the same SQL database. I don't have to pass this, by the way; the table array can be deleted. That means I'm going to be bringing in the kitchen sink—all 256 tables inside of Infinity.
But again, I would like to actually create a vector store based only on these four tables and the relationships between them. At the end, I can use something called `return_raw = True`.
What does `return_raw = True` mean? Well, it's a boolean—true or false—whether to return a plain text dump of the SQL results or parse it into nodes. I would definitely recommend you try it once with true and once with false. If it's true, it's going to just bring in a plain text dump of the SQL, so you'll have to read it however you'd like. But otherwise, it will be nodes coming back. It might be a better way to parse this using a string parser from JSON, for instance, if you would like to do that in a user interface or something like that.
All right, and then finally, I'm going to create a result variable, and I'm going to call in the `NL_SQL_Retriever`. I'm going to say `do_retrieve` and ask my question. What the system is doing right now is taking my question and embedding it, trying to vectorize that specific question into the sphere that has been used for the embedding itself that I already did on the `NL_SQL_Retriever`.
It will be able to find a match and bring me back the results. But remember, I am not asking a question; this is not a query. This is a retrieval. So when I run this, don't get confused and think it will answer the question. This is just trying to create a SQL statement based on what I've done, based on the retrieval that I created for the SQL itself.
So don't confuse this. We're going to finish this up in the next step. I just wanted to show you what will happen when I run that right now. Let me go ahead and press `Ctrl + S` to save this, and I'm going to run it one more time.
Again, it's going to do the same thing it did before. It's going to give me the four users, then it's going to tell me that Leno at the training boss is an admin and a backend user. Finally, it will ask the question, "What roles does the user Kobe at the training boss.com have?"
So let's give it a moment. Let me make this a little bit bigger so you can see it. I have one more thing to show you after that, and I'm hoping that you are enjoying this. While this is finishing, don't forget to like the video, and if you want to see more information in the future and on videos that I'm making, please subscribe to the channel as well.
And there it is! You see the user with the email Leno is an administrator and a backend user. Then there is some mumbo jumbo because I didn't want it back as a node; I wanted it returned raw. So it returned the entire select statement raw in the text coming back itself.
Try it again by saying `return_raw = False`, and you'll get a lot cleaner output. Notice backend users, designer, author. So I can tell that the result coming back from the SQL statement is accurate.
So what happens now if I would like to display this in a chat window? Maybe I'm creating a user interface, and I'd like to show this in a much more professional way. For instance, let's go ahead and do that!