Playbooks
Build a Firm Knowledge Base for AI: Curation, Retrieval and Permissions
A playbook for a law firm knowledge base AI can retrieve from: curating precedents, metadata, permission mapping, platform choice and a 30-question test.
On this page
- Why firm knowledge bases fail before the model is chosen
- Step 1: Decide what goes in, and what stays out
- Step 2: Metadata and chunking that survive legal questions
- Step 3: Map permissions before you connect anything
- Step 4: Choose the platform, five options compared
- Step 5: The 30-question retrieval test
- Ownership, refresh cycles and the audit trail
- What to do this week
A firm knowledge base for AI is a curated, permission-mapped set of precedents and know-how that a model retrieves from before it answers. Building one is mostly editorial work: choosing perhaps 300 documents, not 300,000, tagging them, deciding who may see what, then testing retrieval against 30 real questions before anyone is allowed to rely on it.
Why firm knowledge bases fail before the model is chosen
The failure mode is rarely the technology. An analysis published by Inside Practice in January 2026 concluded that firm retrieval-augmented generation disappoints less because of the retriever than because legal knowledge is fragmented, written for different audiences, walled by ethical screens and unevenly updated. The symptom it describes is precise: "plausible but partial answers" (Inside Practice, Jan 2026).
Two numbers make the point. Stanford RegLab's 2024 test of publisher-built retrieval products found Lexis+ AI hallucinating 17% of the time and Westlaw AI-Assisted Research 33%, and those systems sat on top of professionally curated corpora. A firm's unfiltered document management system, with fourteen versions of every agreement and no marker for which one the partner actually approved, will do worse. Second, when Clifford Chance launched its Knowledge Bank platform with Microsoft and Epiq in July 2026, the headline was not the model but the 400,000-plus documents it had reclassified and summarised first (Artificial Lawyer, 20 Jul 2026).
A 1M-token context window, which Anthropic puts at roughly 555,000 words as of September 2026, tempts people to paste everything in. That works for a single matter. It does not solve the firm-level problem, which is selection and authority, not capacity. Plan on 80% of the effort going into steps 1 to 3 below and 20% into the platform.
Step 1: Decide what goes in, and what stays out
Work one practice group and one document family at a time: Austrian-law share purchase agreements, English-law facility agreements, German employment contracts. The seed is the ten documents the group head would hand a new associate on day one. Everything else earns its place against the criteria below.
| Criterion | Include | Exclude |
|---|---|---|
| Status | Final, signed or firm-approved template | Drafts, negotiation redlines, counterparty mark-ups |
| Currency | Reviewed within 24 months or tagged "evergreen" by the owner | Older material nobody has re-read |
| Audience | Written for the firm's own lawyers | Client newsletters and marketing (different register, different caveats) |
| Authority | Partner sign-off recorded | Associate working notes, unreviewed memos |
| Confidentiality | Cleared for firm-wide or group-wide reuse | Walled material; anything under an NDA restricting internal reuse; anything with an uncleared client name |
| Ownership | A named partner or senior associate | Orphaned documents whose author has left |
A first knowledge base for one group should land at 200–500 documents. If the count is in the thousands, the curation has not happened yet.
The numbered workflow:
- Name the group, the document family and the owner of the project (one partner, one knowledge lawyer or senior associate).
- Collect the ten seed documents and ask the owner what a new joiner gets wrong most often; those errors become test questions in step 5.
- Pull candidates from the DMS by document type and date, then apply the inclusion table.
- Strip client identifiers from templates and positions that will be shared beyond the original matter team, or leave the document out.
- Record owner, status and date in a register before anything is uploaded anywhere.
Step 2: Metadata and chunking that survive legal questions
Retrieval is only as good as the labels. The Vals.ai legal research study of 14 October 2025 found every system tested, including the ones that beat the lawyer baseline, lost roughly 11–14 points on multi-jurisdiction questions. A knowledge base with Austrian, German and English precedents in one undifferentiated pile reproduces that weakness inside your own firm. Tag jurisdiction on every document and instruct the model never to merge them.
| Field | Example | Why it matters at query time |
|---|---|---|
| Jurisdiction and governing law | Austria; England and Wales; Delaware | Prevents cross-border blending |
| Document type | SPA, NDA, facility agreement, board minutes | Narrows retrieval to the right family |
| Party position | Seller-friendly; buyer-friendly; neutral | The "standard" cap on liability differs by side |
| Status and version | Template v4, approved March 2026 | The model must prefer the current version |
| Owner | Partner initials | Someone to ask, someone to refresh |
| Access scope | Firm-wide; group; matter-restricted | Decides copy versus connector (step 3) |
| Supersedes | Link to v3 | Lets the model flag conflicts instead of averaging them |
Put the metadata where the tool will see it: in the file name and in the first lines of the document. A convention such as AT_SPA_seller-friendly_template_v4_2026-03_NS.docx costs nothing and works in every platform in step 4.
Chunking matters only if you build your own pipeline over a vector database; hosted tools chunk for you. If you do build, chunk by clause rather than by page, keep the clause heading and any cross-referenced definition inside the chunk, never split a table, and prepend the document's metadata line to every chunk so a retrieved fragment still knows which jurisdiction and version it belongs to.
Step 3: Map permissions before you connect anything
The moment a document is copied into a shared Project, notebook or vault, it forgets its permissions. Whoever can open the workspace can see it. That is fine for firm-wide templates and positions. It is not fine for matter files, walled material or anything an ethical screen covers.
The alternative is a live connector. The iManage MCP Server, released 14 May 2026, gives AI tools "governed, permission-bound, fully logged" access that respects existing ethical walls and works with Claude, ChatGPT, Copilot, Harvey and Legora (iManage, May 2026). NetDocuments' ndMAX connector runs every call under the user's own credentials, applies matter restrictions automatically, and, in the vendor's words, "documents never leave the platform"; IT enables it with a single admin toggle (NetDocuments, May 2026). The setup steps are in connecting Claude to iManage or NetDocuments with MCP.
The permission map is therefore a two-column decision:
- Copy into the shared knowledge base: firm-wide templates, approved positions, group know-how, anonymised clause banks.
- Connect only: matter documents, walled material, anything with a client identifier, anything whose access list is shorter than the group.
Three further rules. First, use the DMS vendor's own MCP server, not a community one: a June 2026 report found 82% of 2,614 public MCP servers vulnerable to path traversal and 53% relying on static API keys rather than OAuth (Practical DevSecOps, 26 Jun 2026). Second, treat the knowledge base as a memory that can be poisoned: one bad document is retrieved into every answer that touches its topic, which is why the inclusion criteria and ownership in steps 1 and 2 are security controls as well as quality controls; the wider threat model is in agent security for lawyers. Third, for firms in Germany and Austria, the bar rules bite here: BRAK's December 2024 guidance treats tools that store or forward client data for model training as a breach of § 43a Abs. 2 BRAO, and ÖRAK's practice rules require client data to stay on EU servers, so the platform's retention and training terms are a threshold question, not a detail; see ÖRAK, BRAK and CCBE guidance.
Step 4: Choose the platform, five options compared
Facts below are as of September 2026 and pricing where stated is vendor-published or, where marked, from secondary sources.
| Option | Best fit | Capacity and controls | Cost signal |
|---|---|---|---|
| Claude Projects (with Skills) | One group, one document family, a few hundred files, no engineering | Static uploaded context in one workspace; 1M-token context; Skills fire when a prompt matches their trigger; Team and Enterprise data are not used for training | Team $25 standard or $125 premium per seat, five-seat minimum |
| Gemini Notebook (formerly NotebookLM) | Closed-universe Q&A over a defined set; Workspace firms | 50 sources per notebook free, 300 paid, as reported in mid-2026; cannot search the open web; EU data residency for the Enterprise edition since 3 Jun 2026 | Workspace add-on pricing |
| Lexis+ with Protégé Vault | Firms already on Lexis that want firm documents next to Shepard's-grounded research | Vault up to 100,000 documents; Workrooms with dual approvals, role-based permissions and audit trails; customer-held encryption keys | Enterprise pricing on request |
| Harvey Vault and Knowledge | Large firms wanting matter-aware Spaces and Memory (Harvey II, 18 Aug 2026) | Multi-model stack; Agent Builder for group-specific workflows | Estimated $1,000–1,200 per seat per month, 20-seat minimum (secondary sources) |
| DMS connector over MCP (iManage, NetDocuments) | Any firm on those systems; no copying, no second repository | Whole DMS, live, under the user's own rights; single admin toggle; full logging | Included with the relevant DMS tier; check with the vendor |
| Custom RAG (own vector database, chosen model) | Firms with engineering staff and a product owner | Anything you build; anything you break | Engineering time; ongoing ownership |
The Claude Projects route has a known weakness that GC AI listed when reviewing the Claude legal plugin: context resets between sessions unless a profile file is maintained, and single-user markdown files are a poor substitute for team playbooks (GC AI, Jun 2026). Anthropic's Skills, folders with a SKILL.md and optional reference files that fire only when a prompt matches, are the mechanism for making a group's positions reusable across sessions (Layer3 Labs, updated 7 Sep 2026). Set-up detail is in setting up Claude Projects for a matter; the Lexis route is reviewed in Lexis+ with Protégé.
A view on the last row: custom RAG is a product, and products need owners. Latham & Watkins bought its own Nvidia servers to customise open-weight models in September 2026 with around 100 technology staff on AI, and McKinsey's 2026 survey found 38% of professional-services firms now forgo software purchases to build in-house. Below roughly 20 fee-earners in a group, a Project plus a DMS connector beats anything custom. The trade-offs are set out in build or buy legal AI.
Step 5: The 30-question retrieval test
Write the questions before loading a single file, drawn from what associates actually asked the group in the last quarter.
| Set | Count | Example | Pass condition |
|---|---|---|---|
| Lookup | 10 | "Which NDA template do we use for a German counterparty?" | Right document in the top three results |
| Position | 10 | "What is our standard liability cap in a seller-friendly Austrian SPA, and which clause?" | Correct figure, correct clause cited |
| Trap | 10 | "What is our precedent for a Delaware earn-out?" (there is none) or a question whose answer differs by jurisdiction | "No firm precedent found" or a jurisdiction-split answer; never a confident merged answer |
Score each question twice, once for retrieval and once for the answer, on a 0–3 scale: 3 for right document and correct, cited answer; 2 for right document but incomplete answer; 1 for a related document and a plausible-but-partial answer; 0 for wrong, missing or invented. A first build is usable at an average of 2.4 or better with no zeros in the trap set. Most failures at this stage are fixed by metadata and pruning, not by changing the model.
The instruction that sits in the Project, notebook or system prompt should look like this:
You are the knowledge assistant for the [practice group] at [firm].
Answer only from the documents in this workspace.
For every statement, cite the document title and clause number.
If the documents do not answer the question, reply "No firm precedent found for this point" and stop. Do not fill the gap from general knowledge.
Where documents from different jurisdictions could apply, state the jurisdiction of each cited document and do not merge positions across jurisdictions.
Prefer the most recent approved version. If two versions of the same document appear, name both and flag the conflict.
Do not provide case citations or statutory references that are not in the documents.
Output, in this order:
1. Answer in at most 150 words.
2. Sources as a list in the form "Document title — clause".
3. Confidence: high, medium or low, with one sentence explaining why.
Keep the scores in a spreadsheet with a column per refresh date; a score that drifts down between quarters is telling you about stale documents.
Ownership, refresh cycles and the audit trail
Every document has a named owner, and every quarter each owner confirms that their material is still current. Anything not confirmed within two cycles comes out. A removed template is a gap the model will admit to; a stale one is an answer it will give with full confidence.
Ownership is also what separates the firms that see returns from those that do not. Thomson Reuters' Future of Professionals report (June 2026, 1,816 respondents) found 66% of professionals at organisations with a named AI strategy say AI meets expectations, against 22% without one. At scale the knowledge base becomes a team, not a task: Linklaters' internal assistant Laila is used more than 200,000 times a month, and the broader organisational picture is in AI knowledge management for law firms.
Log everything. Queries and answers are records: Anthropic's Compliance API retains session transcripts for six years by default, and the S.D.N.Y. order in the OpenAI copyright litigation treating 20 million de-identified ChatGPT logs as discoverable ESI (affirmed 5 Jan 2026) is a reminder that what a firm's tools recorded can be asked for. Keep the change log, the test scores and the permission map together; they are the evidence that the firm knew what its AI could see on any given date. Module 4 of the AI Lab for Lawyers course, on custom systems, agents and automation, covers the Projects-versus-connector decision with browser-based tools and no code.
What to do this week
- Pick one group and one document family; get the ten seed documents from the group head and write down the three mistakes new joiners make.
- Draft the 30 test questions and their model answers before opening any tool.
- Run the permission map: list what is firm-wide, what is group-only and what is walled; only the first two categories may be copied.
- Load the curated set into the cheapest platform that fits the table above, run the 30 questions, score them, and fix metadata before touching anything else.
- Name the owner, book the quarterly review, and start the change log on day one.
Frequently asked questions
What is a law firm knowledge base for AI?
A curated set of the firm's own precedents, templates, positions and know-how, tagged with metadata and access rights, that an AI tool retrieves from before answering. It differs from a document management system, which stores everything, in that it contains only approved, current material chosen by named owners, so the model cites the firm's standard rather than a random draft.
Do we need a vector database to build a firm knowledge base?
Not at first. For a single practice group with a few hundred documents, Claude Projects, Gemini Notebook, Lexis+ Vault, Harvey Vault or a DMS connector over MCP handle retrieval without any engineering. A custom vector database becomes worthwhile when you need firm-wide scale, your own model choice or integration with internal systems, and only if someone owns it as a product.
How do we stop the AI from seeing documents behind an ethical wall?
Do not copy walled or matter-restricted documents into a shared knowledge base at all. Reach them only through a connector that runs under each user's own credentials, such as the iManage MCP Server or NetDocuments ndMAX, which apply existing ethical walls and matter restrictions at query time and log every call. A copied document forgets its permissions.
How do we measure whether the knowledge base actually works?
Write 30 real questions from the practice group before you load anything: ten lookups, ten position questions and ten traps where the correct answer is that no precedent exists or that the answer differs by jurisdiction. Score retrieval and answer quality on a 0–3 scale, require the traps to return 'no precedent found', and re-run the set after every refresh.
How often should a firm knowledge base be refreshed?
Quarterly, by document owner. Each owner confirms that their templates and positions are still current; anything not confirmed within two cycles is removed rather than left in, because a stale template will be retrieved and used with full confidence. Log every addition, change and removal so you can explain what the model could see on any given date.
Put this into practice
AI Lab for Lawyers: a live, hands-on course for practising lawyers
4 live sessions (8 hours), recordings included, taught by Dr. Niklas Schmidt of Wolf Theiss. Rated 4.7/5 from 381 ratings; alumni from Norton Rose Fulbright, Paul Hastings, Pillsbury and Taylor Wessing.
Related reading
Use Cases
AI Knowledge Management: Turning Precedents into a Firm Brain
AI knowledge management for law firms has four layers: a DMS connection that copies nothing, curation, measured retrieval and permissions. With a 90-day plan.
10 min read
Glossary
Retrieval-Augmented Generation (RAG) — Definition for Lawyers
Retrieval-augmented generation (RAG) searches a document set first and drafts from what it finds. Every grounded legal tool uses it; here is why errors still get through.
4 min read
Playbooks
Connect Claude to iManage or NetDocuments with MCP: A Setup Guide
Connect Claude to iManage or NetDocuments with MCP in six admin steps, test on a read-only matter, enable audit logging and scope permissions before rollout.
11 min read
Tools
Lexis+ with Protégé Review: Workflows, Vault and the Legal Intelligence Engine
Lexis+ with Protégé replaced Lexis+ AI in February 2026 with 300+ workflows, Shepard's-grounded verification and an agentic engine. What it does, costs and still gets wrong.
8 min read
Guides
Build or Buy Legal AI? Latham's GPUs, Harvey's Seats and the Firms in Between
Build vs buy legal AI in 2026: rent Harvey or Legora, assemble on Claude or Astra with MCP, or build like Latham on own GPUs. Costs, control, decision tree.
11 min read
Regulation & Ethics
GDPR and AI Tools for Lawyers: Lawful Basis, DPIAs and Vendor Terms
How GDPR applies when law firms use Claude, ChatGPT or Copilot: controller and processor roles, lawful basis, DPIA triggers, transfers and vendor terms.
11 min read