Transcription
Legal research is one of those domains where AI either has to be extremely precise or it's completely useless. There's no middle ground. Miss a clause, misread a date range, or pull from the wrong contract. And the cost isn't just a bad answer from an LLM. It's a real business problem.
So, when our finance team at Weevate came to us needing help navigating their internal contracts, we built them a full production-ready legal assistant in just 36 hours. So, let's walk through exactly how we did it and why the architecture decisions matter.
If you've built a RAG system before, you know the standard flow. So, a user asks a question, you embed it, retrieve the most semantically similar chunks, hand them back to the LM, and then get an answer. That works fine for a lot of use cases. Legal data is not one of them.
Take this example. Say you ask, "What are the notice periods in our 2024 service agreements?" A basic retriever will pull the most semantically similar clauses, which might be from 2022 contracts because the text looks the same. There's no filter on dates and no awareness of contract type or reasoning about what the question is actually asking for. The system doesn't know what it doesn't know.
Legal queries are multi-dimensional by nature. They require filtering by date, jurisdiction, contract type, counterparty, and a static retriever just can't handle that. You need something that can reason about the search strategy and not just execute on a standard query.
So instead of treating our database as a thing we can query, we can treat it as a set of tools that our agent can reason over. The query agent we have built into Weevate's doesn't just take a user question and then fire off a standard search. It first inspects the schema. So it figures out what collections exist and what structures they have. Then it decides how to break the question down. So a complex query might get split into multiple subqueries targeting different collections. It constructs filters natively, reranks results based on actual relevance rather than just the standard vector similarity, can perform hybrid search or vector search or keyword search, and then synthesizes a grounded answer with cited source passages.
The cited sources is particularly useful for our legal domain. The end user needs to be able to trace every answer back to the clause it came from. This is what makes our system trustworthy and therefore actually useful.
So this is the architecture of the app we're going to build. So the first thing we're going to do is ingest all the legal PDFs using a multi-vector model, specifically Cold-Mortem V-WT with Move Error Compression enabled. So instead of running OCR and chunking all the text manually, this model encodes each PDF page directly as visual tokens, which preserves the layout and tables and other visual information. Mu-Vera can then compress our multi-vector representation into a single vector to keep memory and latency reasonable without sacrificing too much on retrieval quality.
So instead of putting everything into one collection, we're actually going to split the contracts into three different collections. Commercial agreements, corporate IP agreements, and operational agreements. This is going to give the query agent something to route against some sort of structure. So when a question comes in, it can immediately narrow down the search space rather than scanning all of our documents.
Our agent can then operate in two modes depending on what we're trying to do. So search mode focuses on discovery. This means retrieving and reranking the most relevant contract sections without generating an answer response. Ask mode on the other hand synthesizes all the results into a direct answer to the specific question. Results are then streamed back with cited sources which, as we mentioned before, keeps the answers grounded.
So let's build it. First, we're going to install the Weevate agent skills plugin. We can do this directly in Cloud Code with the plugin manager or via npx if you're using Cursor or another coding agent. Once that's installed, we can run the Weevate quick start command. This walks us through creating a Weevate cluster and adding our API keys to the project. It'll take about 2 minutes, but it'll handle all the environment setup. We can initialize everything with an empty collection for now. Don't worry about the data yet. The prompt will handle that.
Now, we can paste in this prompt from our blog post. So, a quick note on why this prompt works and why it has to be so long. The Weevate agent skills plugin automatically pulls in all the Weevate documentation and best practices for building agentic and AI apps as context. So, it can reference cookbooks and architectural patterns without you having to spell everything out. That's the only reason a prompt like this can actually one-shot an app this advanced. The skills carry the context so the prompt doesn't have to.
So the first time I built this legal RAG app, it took a couple rounds of back and forth, specifying the schema, adjusting the ingestion pipeline, getting the front end in a state I was happy with, and a bit of debugging. So then I asked Claude to generate a prompt from that whole process. So what's in the blog post is a distilled version of all that iteration. You're essentially getting all the benefits of a full debugging session for free. You're welcome.
So back to building. Our prompt is running in the background. and it'll download this UAD legal contracts dataset as an example, but you could also use your own data here if you wanted. It'll classify all the PDFs into our three different collections, run the ingestion pipeline, and build both the backend FastAPI app and the front end in Next.js. And after about 12 minutes and 20,000 tokens with Sonnet 4.6, so about 30 cents worth, this is what we get.
So, you can see here we have a full chat interface where you can ask natural language questions across all three contract collections, get direct answers, and see the exact source pages that those answers came from. The front end might need a little polish. I'd maybe clean up a bit how the source pages are displayed and make them a little bit more legible. But the back end is really solid. The ingestion pipeline works. The agentic querying is state-of-the-art. And a finance team could actually just use this out of the box, which is exactly the point.
With coding agents, it's becoming less and less about whether AI can build something and more about whether what it builds is actually useful, reliable, and works in real context. Our finance team is actually using this internally, so I'd say that it passes the test.
What I find most interesting about this project isn't the legal domain specifically. It's what it demonstrates about where RAG and agentic apps are going. Naive RAG is a solved problem for simple use cases, but as soon as you're dealing with structured data, complex queries, or domains where precision actually really matters, you need a reasoning layer on top. This is one of the most exciting things enabled by agents. Reasoning on how to deal with complicated data in a precise way.
If you want to build this yourself, the blog post with the full architecture and the prompt is linked in the description. And if you have questions about any of the technical decisions, the multi-vector setup, the collection schema, why we use MOA, feel free to drop them in the comments.