Transcription
Let's talk about context URL in Gemini API, which is one of my favorite and probably the least utilized Gemini tool. It helps Gemini ground its answers in text, PDF, and image files that are available on the internet. There are dedicated tools like reader API from Gen File Crawl and even crawl for AI, which is an open-source project for web scraping. All of them are great options. However, I have found that the URL context seems to be a much faster and more reliable approach.
Now, here is how it works. It's a two-step process approach that finds a balance between speed, cost, and access to fresh data. Since Google is one of the biggest search engines, if you provide a URL as a context, it first looks at the indexed cache. Now, if the URL is not cached by default, then it's going to fall back to live fetching of the data. Initially, it was only text, but now they have added support for images, and my favorite is PDF file parsing. So, it can read PDF files, and since Gemini has document understanding capabilities, that means that it's not parsing PDFs into markdowns and then processing it. It is actually looking at every page visually, which I have found to be extremely useful for my own applications.
Okay. So, in this video, I'm going to walk you through how to set this up and how to use this through the Gemini API. If you are in AI Studio, you can directly enable this using the URL context switch or toggle. Doing that will basically add it to the context of your large language model. So, now you can just provide a URL, and it will be able to use that for grounding its answers. But I'm mostly interested in the API use case, and here are the models that support this specific tool. So, in the rest of the video, I'll show you an example of how to set this up with the API. But before that, let's talk about some limitations.
So, it's not available through the traditional function calling. You can still use this as a tool. There currently is a rate limit of 20 URLs per request, where a single URL can have up to 34 megabytes of data, and you can only access publicly accessible URLs on the web. Now, one more thing I really like about this is that you don't need any external scraper anymore if you know the URLs, and you are not being charged for any extra service. Essentially, these are the number of tokens to the model from scraping the URL or website that you're providing. Now, the good thing is that it's not just a raw scraping of HTML files. It's actually a well-formatted output like Markdown or a direct scraping of PDF files.
Okay. So, in the rest of the video, I'll walk you through this notebook, which is going to show you an example of how to use grounding with Gemini API and also use the URL context. But before that, a quick word from today's sponsor because somebody has to pay for the bills. Today's sponsor is Outskll, which is an AI-focused educational platform to take your AI skills to the next level. They're hosting a 2-day generative AI engineering mastermind, which is going to be live this Friday, 10:00 a.m. to 2:00 p.m., and Saturday, 10:00 a.m. to 7:00 p.m. Eastern time. As an early access, you can sign up for free in the next 48 hours for this workshop. In this workshop, you're going to learn how developers can build AI agents and boost their productivity, how to build end-to-end agentic workflows, and even build a team of AI agents from scratch. This is going to be a hands-on workshop where you're going to learn from industry experts. If you attend both these, you are going to get free bonuses such as the AI engineering prompt bible, ready-to-implement custom GPTs, and access to high-profile GitHub repos and templates. Seats are limited. Use the link in the description to join. And just when you sign up, you'll get access to their 2026 AI survival handbook. So, join their free WhatsApp community as well. Or you can just scan this QR code for more details. Thanks to Outskll for sponsoring this video. Now, back to the video.
So, for this, you need to have access to the Gemini API key. You can get your free API key from AI Studio. We will need to install the latest version of the Google generative AI Python package. A quick side note for all the developers: if you are building an application, I highly recommend to use the REST API instead of any of the SDKs because that will reduce the complexity that you are dealing with. Normally, the SDK changes will require you to update your codebase, but if you're just using the REST API, that's going to save you a lot of headache.
Next, we're just setting up our API key. Initializing the SDK. For this, we're going to be using Gemini 3 Flash preview. Now, the first example is grounding with Google Search. Now, for this new API structure, here's the format that you want to provide. So, you have the configuration tools, and then you simply list the tools. Now, if you are using some of the built-in tools through the Gemini API, like Google Search, code execution, or even the context URL, you can just provide them here. Now, this is a quick example of how you're going to structure your query if you're using one of these built-in tools. In this case, Google Search is our built-in tool. Now, based on the query, the agent or your model is going to generate inputs for this specific tool. In this case, we are getting up-to-date information because it is using the Google Search for grounding. Now, if you don't provide the tool to the agent or the LLM, it's going to just use its own internal knowledge that is frozen during training, and you can see that it is providing outdated information.
But let's talk about grounding with the URL context. It's very flexible. As I said, it gives you the ability to parse data from HTML files, PDFs, and you can directly provide image URLs; it will be able to process those. It also supports JSON, XML. So, it's a very flexible inverse style. So, the structure of the tool configuration is very similar to what we saw from grounding with search. The first example in this case is going to be grounding your answers in a specific URL data. As you can see, we're providing the URL directly in the prompt and just asking it a question. Now, since it has access to this URL context tool, it can generate the input for this specific tool. The context in this case was a web page from the Google DeepMind Gemini models. There are expandable elements in this specific URL, and the model is able to retrieve information from all these elements. So, it generated a pretty nice-looking table that has all the key information that we were asking for, and basically, it looked at all those expandable components or elements that are present on the URL and retrieved information from there. Also, it is giving us key differences for these different model families. And in here, you can actually see what were the exact links that we used. Now, if it has already indexed this specific web URL, it's going to use the indexed version. Now, if it's a fresh URL, then it's going to retrieve the live data.
Okay. So, a couple of other examples. In this case, we are providing a link to a PDF file and asking questions on that specific PDF file, and we get answers based on the contents of that file. I also tried this on the "Attention Is All You Need" paper, just directly providing that and asking about figure number one. It seems to be able to read through that specific figure. Not only that, you can actually provide an image URL. In this case, we have this image, and the prompt is, "Can you help me name the numbered parts of this instrument in French?" So, on the image, we have eight different parts, and we get responses for all of them. Now, if somebody knows this musical instrument plus the French, do let me know if the responses are actually correct.
Now, the beauty is that you can combine different tools. So, in this case, we are combining the Google Search plus the context URL tool, and now the model will be able to retrieve data based on both of these tools and generate an answer.
Okay. So, this was a quick video on something that I use a lot, and I thought it will be extremely helpful for the community as well. You don't really need dedicated scraping services anymore. If you're using Gemini API, I'll highly recommend to explore the URL context tool in the API. Anyways, do let me know if you have a specific use case for this. I hope you found this video useful. Thanks for watching, and as always, see you in the next.