AI Agents for Lawyers

Glossary

Vector Database — Definition for Lawyers

A vector database stores text as numerical embeddings so an AI system can find passages by meaning, not keywords. Its role in RAG and knowledge management, and its limits.

By Dr. Niklas SchmidtUpdated 15 November 20263 min read
On this page
  1. How embeddings work
  2. Semantic search versus keyword search
  3. Where vector databases fail lawyers
  4. Questions to ask a vendor
  5. Where this leaves you

A vector database stores text as embeddings, lists of numbers that represent what a passage means, and retrieves the passages whose meaning is closest to a question. It lets an AI system find the indemnity clause that answers your query even when the clause uses none of your words. In legal tools it usually sits behind retrieval-augmented generation, deciding which passages the model gets to read.

How embeddings work

An embedding model reads a passage and outputs a vector: a long list of numbers, typically hundreds or thousands of them. Passages with similar meanings produce vectors that sit close together in that numerical space; unrelated passages sit far apart. "The seller shall indemnify the buyer" and "the vendor will hold the purchaser harmless" land near each other even though they share almost no words.

A vector database is built to store millions of these vectors and to find, very quickly, the ones nearest to a new vector. The pipeline in a typical system:

  1. Chunk. Split each document into passages, for example by clause or paragraph.
  2. Embed. Turn each chunk into a vector with an embedding model.
  3. Store. Save the vectors in the database with a pointer to the source document, page and any metadata such as matter, date or access group.
  4. Query. Embed the user's question the same way and retrieve the nearest chunks.
  5. Generate. Pass those chunks to the language model with the question, and require it to cite them.

The guide to how large language models work covers the model side of step 5.

Keyword searchVector (semantic) search
MatchesThe exact words typedPassages with similar meaning
Finds a reworded clauseNoYes
Finds a defined term, section number or citation exactlyYesNot reliably
Explains why a result matchedYes, the term is highlightedNo, closeness is a number
Typical legal useDocket numbers, party names, citationsPrecedent search, clause finding, Q&A over documents

Neither is enough on its own for legal work, which is full of both paraphrase and precise identifiers. Systems that combine the two, often called hybrid search, are the usual answer.

Where vector databases fail lawyers

Chunking errors. If a clause is split from its heading, or a definition from the clause that uses it, the retrieved passage can be misleading on its own. The knowledge-base playbook recommends chunking by clause and keeping headings and cross-referenced definitions with each chunk.

Silent misses. Retrieval returns the closest passages, not all relevant ones. If the decisive provision is not retrieved, the model answers without it, confidently. That is a route to hallucination by omission, and it is why answers need pinpoint references you can check against the full document.

Access and confidentiality. An index built from the DMS is a second copy of client data. If it ignores ethical walls or matter-level permissions, anyone who can query it can surface walled-off material. The knowledge management use case explains why querying the DMS in place, with the user's own permissions, is often the safer design.

Staleness and deletion. When a document is updated or must be deleted, its vectors must be updated or deleted too. Ask how the vendor handles both.

Questions to ask a vendor

1. How are documents chunked, and can a chunk split a clause from its heading?
2. Is search semantic, keyword or hybrid?
3. Do matter permissions and ethical walls apply at query time?
4. Where is the index stored, and is it covered by the same data terms?
5. How quickly are edits and deletions reflected in the index?
6. Does every answer cite the source passage with a page reference?

Where this leaves you

You rarely need to build a vector database, but you need to understand one to judge the tools that do. Module 1 of the AI Lab for Lawyers course covers search, RAG and how retrieval shapes what a model can answer.

Frequently asked questions

What is the difference between a vector database and keyword search?

Keyword search matches the words you type, so a search for 'termination for convenience' misses a clause that says 'either party may end this agreement without cause'. A vector database compares meaning, so it can find that clause. It is weaker at exact matches such as a defined term, a section number or a case citation, which is why most serious legal search tools combine both methods.

Do lawyers need to build a vector database to use AI on firm documents?

Usually not. Most legal AI products and enterprise assistants build and manage the index themselves, and some query the document management system in place. Building your own makes sense only with technical support and a specific need. What lawyers should ask any vendor is how documents are chunked, whether access rights carry over to search, and where the index is stored.

Put this into practice

AI Lab for Lawyers: a live, hands-on course for practising lawyers

4 live sessions (8 hours), recordings included, taught by Dr. Niklas Schmidt of Wolf Theiss. Rated 4.7/5 from 381 ratings; alumni from Norton Rose Fulbright, Paul Hastings, Pillsbury and Taylor Wessing.

Reserve a seat · €200 Maven money-back guarantee · certificate of completion