Glossary
Retrieval-Augmented Generation (RAG) — Definition for Lawyers
Retrieval-augmented generation (RAG) searches a document set first and drafts from what it finds. Every grounded legal tool uses it; here is why errors still get through.
On this page
Retrieval-augmented generation (RAG) is a technique in which an AI system first searches a defined body of documents for passages relevant to your question, then hands those passages to the language model together with the question, so that the answer is drafted from retrieved text rather than from the model's memory. Every "grounded" legal research tool, from CoCounsel to Lexis+ Protégé, relies on it.
How it works, on one research question
Take a question a litigator might ask a research tool: whether an English landlord must serve a section 146 notice before forfeiting a commercial lease for non-payment of rent. Without retrieval, the model answers from whatever it absorbed in training, and if it never absorbed the right case it produces the case most likely to exist. With retrieval, five things happen in sequence:
- Search. The retriever turns the question into a query. That may be keyword search, or an embedding compared against a vector database of the corpus, or both. It returns the top matches: perhaps twenty paragraphs from judgments and a practice note.
- Assembly. Those passages are placed in the model's context window with an instruction along the lines of "answer only from these sources and cite them by paragraph".
- Drafting. The model writes the answer with citations to the passages it was given.
- Verification layer. The better products then check the cited authorities against a citator: Lexis+ with Protégé describes Shepard's-grounded verification, Thomson Reuters ships Deep Research Verify in CoCounsel Legal.
- Presentation. You see an answer with links to the sources. The links are real because they came from the search index, not from the model.
Two consequences follow. The model never had to remember a case, so inventing one from nothing becomes rarer. And every one of the five steps can fail silently.
Where lawyers meet it
| Product (as of September 2026) | What it retrieves from | What the vendor says about grounding |
|---|---|---|
| CoCounsel Legal (Thomson Reuters) | Westlaw | Deep Research Verify; next-generation version GA in the US on 20 August 2026 |
| Lexis+ with Protégé | Lexis content | Shepard's-grounded verification; US GA 24 February 2026 |
| Astra for Law (OpenAI) | Legal Search Index of 230 million-plus URLs of US law, built with CourtListener and the Free Law Project | 54% on the Vals Legal Research Bench against 38.7% for GPT-6 with plain web search (OpenAI-reported, September 2026; treat as days old) |
| Harvey Vault and Knowledge; Legora Tabular Review | Your uploaded matter documents, up to 100,000 in Legora's case | Tabular output with per-cell citations |
| Gemini Notebook (formerly NotebookLM) | Only the sources you upload, 50 free or 300 paid | Closed universe; cannot browse the open web |
| Claude Projects, a firm knowledge base | Files you or the firm curate | No citator; you supply the corpus and the verification |
The last row is the one most firms end up building themselves, and the firm knowledge base playbook covers the curation and permission questions that decide whether it works.
Why grounding cuts errors but does not end them
Stanford's RegLab put the vendors' "hallucination-free" claims to the test and measured 17% hallucination for Lexis+ AI and 33% for Westlaw AI-Assisted Research on May 2024 versions, against 43% for ungrounded GPT-4. Better, not fixed. The sanctions record says the same: the 16 fabricated quotations in Fletcher v. Experian (5th Cir. 2026) were traced to vLex and CoCounsel, and the $31,100 in Lacey v. State Farm (C.D. Cal. 2025) followed a brief built with CoCounsel, Westlaw Precision and Gemini. A 2026 academic benchmark of legal RAG still finds state-of-the-art systems hallucinating at the level of individual claims.
| Failure mode | What happens | How you catch it |
|---|---|---|
| Retrieval miss | The leading authority is not among the top passages, so the model answers from what it was given: a plausible but partial answer | Run your own search for the case you expected to see; ask the tool what it excluded |
| Misreading | The retrieved case is real but the model states a holding it does not contain; on the HAQQ benchmark (June 2026), 24% of answers misapplied the law they cited | Open the source and read the paragraph; nothing else works |
| Stale or walled corpus | Firm RAG fails less because of the retriever than because legal knowledge is fragmented, walled by ethical screens and unevenly updated (Inside Practice, January 2026) | Ask when the corpus was last updated and what your permissions exclude |
| Fabricated reference despite search | A University of Pennsylvania study (April 2026) found search-augmented deep-research agents fabricating 3% to 13% of cited URLs | Click every link before it goes in a filing |
| Over-refusal | In Vals' February 2025 study, Vincent AI "refused to answer, rather than hallucinate"; the safer failure, and still a failure | Rephrase, narrow the jurisdiction, or search manually |
The practical shift is this. An ungrounded chatbot makes you ask whether the case exists. A grounded tool makes you ask whether the case says that, and the hallucination statistics show the second question is where lawyers now get caught. The workflow for asking it properly, tool by tool, is in the AI legal research page; Module 2 of the AI Lab for Lawyers course covers legal research and document analysis, where grounded tools do most of the work.
Frequently asked questions
Does RAG stop AI hallucinations?
No. It reduces them, mainly the wholesale invention of cases, because the model is drafting from passages in front of it. Stanford's RegLab study still measured 17% hallucination for Lexis+ AI and 33% for Westlaw AI-Assisted Research on 2024 versions, and a 2026 academic benchmark of legal RAG systems finds claim-level errors in state-of-the-art systems. The residual errors are real cases misread, which are harder to spot than fakes.
What is the difference between RAG and fine-tuning?
Fine-tuning changes the model itself by training it further on examples, which is expensive and freezes knowledge at training time. RAG leaves the model unchanged and supplies documents at the moment of the question, so the corpus can be updated daily and every answer can point to its source. Spellbook grounds frontier models rather than fine-tuning them; Harvey's Tenet is a post-trained legal model. Most legal products use both.
Is Gemini Notebook (NotebookLM) a RAG tool?
Yes, in its purest form. It answers only from the sources you upload, 50 per notebook on the free tier and 300 on paid plans, cites the passage it relied on, and cannot search the open web. That makes it a closed-universe tool: reliable for questions about the file in front of it, useless for finding authority that is not in the notebook. Use it in a Workspace or enterprise account, not a personal one.
Put this into practice
AI Lab for Lawyers: a live, hands-on course for practising lawyers
4 live sessions (8 hours), recordings included, taught by Dr. Niklas Schmidt of Wolf Theiss. Rated 4.7/5 from 381 ratings; alumni from Norton Rose Fulbright, Paul Hastings, Pillsbury and Taylor Wessing.
Related reading
Glossary
Hallucination (AI) — Definition for Lawyers
An AI hallucination is fluent, confident, false output: an invented case, a misread holding or a fake quote. 2,041 court decisions now involve one. What to know.
5 min read
Glossary
Vector Database — Definition for Lawyers
A vector database stores text as numerical embeddings so an AI system can find passages by meaning, not keywords. Its role in RAG and knowledge management, and its limits.
3 min read
Playbooks
Build a Firm Knowledge Base for AI: Curation, Retrieval and Permissions
A playbook for a law firm knowledge base AI can retrieve from: curating precedents, metadata, permission mapping, platform choice and a 30-question test.
11 min read
Use Cases
AI Legal Research in 2026: What Works, What Hallucinates
AI legal research now beats a lawyer baseline on accuracy yet misapplies law in 24% of answers. Which tools are grounded, where errors hide, a routine that holds.
11 min read
Tools
CoCounsel Legal (Thomson Reuters) Review: Westlaw-Grounded Agents
CoCounsel Legal reviewed as of September 2026. The Claude-built rebuild, Brief Builder, Tabular Analysis, Deep Research Verify, $104 to $639 pricing and grounding.
8 min read
Comparisons
Legal-Specific AI vs General-Purpose AI: Do Lawyers Need a Vertical Tool?
Legal-specific AI vs general-purpose AI: why legal tool use fell to 40%, what Harvey, CoCounsel and Legora add over Claude or ChatGPT, and a decision by task.
10 min read