Transcription
Okay. So in the last video, we looked at an introduction to Pydantic AI. Since then, I've been playing with it quite a bit. I got to say, it's really quite nice for building things.
So what I wanted to do was a couple of follow-up videos. I've got another one on RAG coming, but what I want to do in this one is build a research agent. Lots of people have kind of done things like Perplexity, that kind of thing. This is not exactly like that, but it's going to show you a lot of the elements that you can use for that kind of thing and how Pydantic AI makes those things really easy.
So, similar sort of setup to last time. I'm basically just got a simple notebook. I'm doing it in a notebook just so everyone can load this and use it straight away, rather than have to set up their own environment, et cetera.
Okay. Once you install the different things in here, you need to restart the notebook. We talked about that last time. Another thing in here is I've set up two different search engines. I've set up Duck Duck Go search, which is the one that I would probably normally show more often. The challenge with that at the moment, though, is definitely they seem to be rate limiting searches.
So if you've got a proxy system, you can use that with Duck Duck Go, and that works quite nicely. The other alternative is to use Tavily, which I've got in here. So just showing you those two quickly, Duck Duck Go search is pretty easy to do. There's a nice unofficial Python package for this. It supports both synchronous and asynchronous calls.
In this particular agent, we want to use asynchronous calls, and it's got a few different types of search. The same is true for Tavily. If you basically set this up with the client being async, then everything will run in an async manner in here.
You can see that, okay, you can do sort of a basic search where we pass in the max results, and then we can see that we get three results back. Or they have what's called a sort of a RAG context search, which we'll also go through and get probably more detailed responses back.
What I recommend is you pick one of these. The cool thing with Tavily, even though it's a paid service, is you can get an API that gives you a thousand free calls a month. So you can certainly get started for nothing. You don't even need to put a credit card in or anything like that. They're quite reasonable in that way.
So you've got a choice. You can either use the Duck Duck Go search one or the Tavily search one. I'm going to go with the Tavily search one just so we don't have any rate limits or anything like that going on in here.
All right. So what I want to show you in here is building an agent where we're going to do a number of dependencies in this. You'll see that we're bringing in a number of different things from data classes and some Pydantic stuff as well.
What we're going to do is have a search data class, which really I'm just going to be using for max results. This is what we're going to pass through the agent to actually go to the search tool. The other thing we're going to have is a research dependency. So I'll come back to this. I'm not going to use this to start; I'll use this later on and I'll explain why.
The other big thing is we're going to use a result type. This is basically a data class, a Pydantic data class, not actually Pydantic AI, just the original Pydantic. In this, we're going to tell it what we want in the final result.
So in the final result here, I want to have a research title. I want to have sort of a research main bit, and then I want to have some summaries at the end. Each of these is going to be strings. I want them to return in markdown, although you'll see later on that we have a bit of challenges with the heading title one.
But you can see, for example, for the main, this is going to basically say, this is the main section that provides the answers for the query and research. For the bullets, this is a set of bullet points that summarize the answers for the query in here.
All right, next up, I'm going to actually set up the agent itself. In this case, I'm just using OpenAI GPT-4o, but you could play around with different models, right? This is the cool thing; it's literally to take this from GPT-4o to GPT-4o-mini or to Gemini, et cetera. We can just swap this out and it will take care of the rest on the back end for us.
And while they don't have Anthropic yet, it's supposedly on the way. So hopefully, we'll get that in the not too distant future. All right. So you're going to see here in my setting up the agent, now I'm passing in these research dependencies and this result type.
So let's focus on the result type first. As I talked about here, this is basically, you're just going to define what the sort of structured output is going to be in there. We've then got our system prompt. So here, we've got your helpful research assistant. You're an expert in research. If you were given a question, you write strong keywords to do three to five searches in total.
So the agent itself is going to decide how many searches to do, each with a query number. I want to show you passing; we're going to pass the query number through so that we can actually see what queries it decided and stuff like that, and then you're going to combine the results.
Now, I can certainly elaborate more on this system prompt, right? This is a very simple system prompt here. I could, for example, if I wanted a research thing to be academic, I could say, you focus on academic things. If I want it to be more commercial, where it's checking prices or something, I could load all those kinds of things into this system prompt in here.
Because this is going to be used to basically determine the searches and then determine how it filters those to get our response back. All right. So next up, we need to actually make the search agent tool. If you look back here, we've got this search agent tool.
The main thing we're going to pass in here is the max results. So creating the tool, we basically have got a decorator that's going to be at search agent dot tool. In this case, this is the one I've commented out, the Duck Duck Go search one. You could put that in; don't put both. It's probably going to confuse things there.
But you can see that, okay, how's it going to know what this tool is? We've got a doc string for that, right? So this is going to get the search for a keyword query. And then the query is going to be the keywords to search in here.
So we've got the query that we pass in; that's going to come from the model. And we've got the query number that we referred to up here. That's going to come in as well. So these are decided by the LLM. We've then got the search data class, which is really defining the dependencies for this.
So I'm going to constrain it so that for each, you'll see later that I'm going to constrain it so that for the max results, I'm going to pass that in at query time. You could imagine that if we had some kind of UI and we had a dropdown of, okay, how many results do you want me to get for each of these queries? We could pass that in there.
Now we're going to just show what the search query is and the number that's associated with that. So these two are obviously coming from the LLM there. Then Tavily is going to make a search. It's going to do a get search context. It's going to pass in the query. It's going to pass in the max results that it got from the dependencies that we've put in there.
Same kind of thing for the Duck Duck Go in here. All right. Now, setting up the dependencies, you can see that, okay, I want to have the date, which I'm not going to use at the start, but we're going to use it later. I'm going to show you how you would even inject that into the system prompt.
But we can basically make a date string, and we can pass that either into the search data class. Actually, where I'm going to end up using it is not in here. I'm going to end up using it in the system prompt. I found that to be an easier way to be able to do it, and it also shows you how to update the system prompt on the fly.
That's why I've put it in there. You could put it in here and then append it to the query or something like that if you had a very special way that you wanted it appended to the query. One of the nice things with the way I'm going to show you is that if it doesn't need it, it will just not inject it in there. The LLM can decide that itself.
All right. We've got these dependencies, though, the key one here being the max results. All right. So I've basically set these up, and now I'm just going to inject my query into this. I've got my search agent, and I'm just going to basically say, can you give me a very detailed bio on Sam Altman?
We can see in this case, it's decided to do four queries as it's gone through this. Now you notice that each of the queries is not waiting for the previous query to go, right? It's all async in this case. And we can see that the queries that it decided.
Now, remember, it could choose between three to five queries, and then for each query, it's still getting three results back. So we're getting quite a lot of tokens back as we look at this. So if we look at here, we can see that, okay, it decided to go for search query, Sam Altman biography, early life, Sam Altman career, OpenAI, Y Combinator, Sam Altman achievements, contributions, personal life, and philanthropy.
All right. If we come in here, we look at the data. We can see that sure enough, this data has come back. It's gone and got all that data. It's gone back to the LLM to format it into a response for us.
And our response needs to have a research title. It needs to have the research main, and it should have somewhere in there research bullets as well. So let's look at these as we go through. I'm going to append the markdown heading just because I got frustrated with trying to get the right prompt to actually put that on.
It may be actually doing some kind of filtering out. I'm not sure why it wouldn't just automatically put that on. But anyway, you can see that we've got our research title there. If we look at the research main, we've got that. And if we look at our research bullets, we've got that as well.
So we've now got through like this structured output, nicely of okay result.data dot research title, result.data research main, result.data research bullets. Okay. If we combine those into markdown and print it out, we can actually see that, okay, we're getting out a sort of full-on markdown response.
Now, each time you'll run, it'll be different. Sometimes you will actually get subheadings in here, and we could have actually asked for that. We could have said, okay, break it up into subheading one, research main part one, subheading two, research main part two, et cetera.
We can structure this how we want it to come out. That's one of the cool things in here. Now, if we come in here and actually look at the result, the whole thing that got sent back and actually what happened through this, we can see sure enough that we created our agent.
Our agent had a system prompt. We had a user prompt, and then we had the different queries. This was basically query one being sent to the search tool, query two being sent to the search tool, query three, query four, et cetera.
One of the cool things with doing it like this is that if I ask it, just give me a bio of Sam Altman without saying detailed or any of that sort of stuff, it'll probably just do three query researchers. Actually, most of the testing when I was doing this, asking for very detailed, it's actually giving me five.
So the cool thing here is the agent is deciding itself, okay, this topic, what would be the sort of key keyword searches to do in here? And I can do between three and five of those. All right. We can see then we're getting our tool returns back for each of these.
So we've got four of these different things. Each of them is going to have three responses back. You can see on this one it's very clear. We've got, this is one response. The next one is another response, et cetera, coming back. It's clear that we're getting a lot of tokens in here. That's the key thing.
And then now it goes back to the model, and then the model now is then using these to come out with this structured response that's coming back for our final results. So the final result in here is going to have our research title. It's going to have the research main that we've got there, and then somewhere down here is going to be the research bullets.
So the cool thing is if we decided, oh, actually we want it to write a 500-word essay or something like that, and we wanted it structured in a very particular way, all we have to do is go back to the result type class and just flesh that out more.
We could change that to be a whole bunch of different things that fit in the Pydantic schema, et cetera, for going through this. So in the end, this is what we get back, actually. So it's interesting here that it did actually give us back the hashtag, but maybe it's getting filtered out some way.
Anyway, we can see the cost of how many tokens that we've used in here. Total tokens, 3,700. And obviously, we could change it and even put information in the system prompt about how it should be economical and it shouldn't use more searches than are needed because they cost money or more tokens than are needed because they cost money.
All of those sorts of things that will respond to pretty well going through here. All right, next up, I want to show you something.
All right, what happens if we take that exact same agent? And we say, hey, what is the latest new AI news? So now it's basically purely relying on, it understands that, okay, I need to get the latest news, so I probably need to give the searches a date.
But it doesn't have a date in its system prompt or anything like that. So it's taking the date of the cutoff period. So we can see that's October 2023. And so it basically comes back with latest AI news, October 2023.
And if we print that out, if we look at this from an earlier version, we can see that clearly we're getting things out of date. There's nothing new in there. So what if we add in the date?
So to do that, we can just inject into the system prompt, so we could just override the system prompt. And we're doing that with the research dependencies. Now the research dependencies is the only one that we had was today's date.
And we set that up above where we got it from date time; we converted it to being date, et cetera. Now we've got, you're a helpful research assistant. You're an expert. We've got everything the same, except we say, if you need today's date, it is.
And then we pass it in, and that's going to be in this format up here that we can see that we've got the year, month, day in the string there. All right. We inject that in. And now we do the, we ask it, okay, what are major AI news announcements in the last few days?
And it decides in this case that it only needs three search queries, right? AI news announcements, December 2024, possibly because it realizes that we're only in the first week of December. AI, recent announced AI developments, latest AI advancements.
So it comes back with all our data just like before. We get this, we get our research title, research main, et cetera. If we go through and look at those, we can see what they are. And then if we print this out, we can see that, okay, sure enough, it's come back with some stuff.
And the bullets are basically saying, OpenAI announced 12 days of Shipmas events to reveal new AI products, and we can see rumors about OpenAI launching ChatGPT pro plan at $200 and text2video SORA model.
So it's got most of those things, and it's got some other things as well. You could imagine that we could ask it to focus on announcements by company, this company, that company, these sorts of things, and we could inject those into the system prompt as well as we're going through this.
But now we've got a proper response that is matching what we would expect for something that when we're doing this kind of query today, as opposed to not having that today's date and then losing it there.
And if we come in here and look at this, we can see sure enough, it starts with the original system prompt, but then the system prompt gets overridden to basically have the date in here in the year, month, day format for this.
So this is a reasonably simple research agent. We've got the one agent doing lots of different calls in there. We could add in different tools if we wanted to have different kinds of searches for different kinds of topics or something like that.
If it's a search about a person, use this search engine; if it's a search about a company, use this, that kind of thing. We could put that in there to make it so we could get better responses out of it, but it just shows you the fundamentals of structuring the output so that you're going to basically get results back in a way that's going to be useful to you.
Structuring the dependencies so that you can inject different things into the agent as you're using it, and just how to set up some of the tools and stuff like that.
So in the next video, we'll probably look at doing something similar to this with a RAG system, which overall is probably going to be very similar. We can then decide, okay, how we want to deal with the context and stuff like that, and we're just incorporating a vector store, et cetera, in there.
Anyway, as always, if you've got any comments or questions, please put them in the comments below. If you found the video useful, please click like and subscribe. And I will talk to you in the next video. Bye for now.