AI Agents for Lawyers

Glossary

Large Language Model (LLM) — Definition for Lawyers

A large language model is a neural network trained to predict the next token, and it powers every legal AI tool. How LLMs are built, the models of September 2026 and what follows for reliability and confidentiality.

By Dr. Niklas SchmidtUpdated 13 November 20264 min read
On this page
  1. How an LLM is built, in three stages
  2. The models lawyers meet in September 2026
  3. Two consequences of the mechanism
  4. Three habits that follow

A large language model (LLM) is a neural network trained on a vast body of text to predict the next token, and through that single skill to draft, summarise, translate and reason. It is the engine inside Claude, ChatGPT, Gemini and every legal AI tool sold in 2026. It produces probable text, not retrieved fact, which is why it writes fluently and cites carelessly.

How an LLM is built, in three stages

  1. Pre-training. The network reads trillions of tokens of text and adjusts billions of internal weights until it predicts the next token well. Nobody labels the law; the model absorbs the statistics of language, legal language included, from the corpus. Its knowledge stops at the training cutoff: June 2026 for Claude Fable 5.1, May 2026 for Claude Opus 5, 30 April 2026 for GPT-6 Astra. Anything later must be supplied in the prompt or fetched by a tool.
  2. Post-training. The raw predictor is shaped into an assistant: instruction-following, refusals, tone and, increasingly, the ability to plan and call tools. This is where vendors differentiate. Anthropic says Claude Opus 5 "verifies its own work without being told to" and that its first-turn NDA redlines scored "nearly double Opus 4.8" (Anthropic, 24 July 2026).
  3. Domain work, optional. Harvey released Harvey Tenet, a post-trained open-weight legal model, as a research preview on 20 August 2026 (Harvey). Spellbook takes the opposite view and grounds frontier models with retrieved documents instead of fine-tuning, its chief executive arguing that fine-tuning "encourages hallucinations" (Nonbillable interview). Both approaches ship.

The models lawyers meet in September 2026

ModelReleasedAPI price per million tokens (in / out)ContextNote for legal work
Claude Fable 5.1 (Anthropic)1 Sep 2026$10 / $501MNo. 1 on the Vals Index (15 Sep 2026); requires 30-day data retention; not on the Free tier
Claude Opus 5 (Anthropic)24 Jul 2026$5 / $251MDefault on Max; live in Harvey from launch day
Claude Sonnet 5 (Anthropic)30 Jun 2026$2 / $101MDefault on Free and Pro
GPT-6 Astra (OpenAI)4 Sep 2026$10 / $50~1.05MBase of Astra for Law; off by default in ChatGPT Enterprise; "recurrent depth" hides part of its reasoning
Gemini 3.1 Pro (Google)19 Feb 2026$2 / $121MFlagship Pro model; Gemini 3.5 Pro announced 19 May 2026 and still unreleased in late August
Muse Spark 1.3 (Meta)2 Sep 2026—1MThe Max version tops the Vals Legal Research Bench (15 Sep 2026); the cheaper "contributor tier" retains your data
DeepSeek V4-Pro13 Aug 2026Open weights, MIT licence1MBanned on US Department of Defense and Australian government systems

Sources: Anthropic model overview, Vals benchmarks, GPT-6 Astra.

Most legal products are wrappers around these engines rather than engines of their own. Harvey runs a multi-model stack (Opus 5 live 24 July 2026, Fable 5.1 from 1 September), Thomson Reuters rebuilt CoCounsel Legal on Anthropic's Claude Agent SDK plus its own legal LLM (US general availability 20 August 2026), and Lexis+ with Protégé draws on models from Anthropic, Google and OpenAI. What the vendor adds is retrieval, verification and workflow; the model underneath is one of the rows above. The Anthropic line is reviewed in Claude for lawyers; the mechanics are in how large language models work.

Two consequences of the mechanism

Reliability. A next-token predictor produces a plausible citation exactly as readily as a real one; there is no separate check-the-reporter step unless a tool adds it. HAQQ's benchmark of 3,000 answers across 51 practice areas (5 June 2026) found that 24% of all answers cited or misapplied law that did not support the claim, and that every model, the leaders included, fabricated or misapplied a citation somewhere (HAQQ). Damien Charlotin's database of court decisions involving fabricated AI output reached 2,041 on 14 September 2026, 811 of them involving lawyers (Charlotin). The definition of hallucination follows directly from the definition of an LLM.

Confidentiality. The weights do not store your prompt in any retrievable sense (the Hamburg data protection authority's July 2024 discussion paper takes this view and locates the GDPR risk in inputs and outputs), but the platform around the model may. Consumer ChatGPT trains on conversations unless the user opts out; consumer Claude users had to choose by 8 October 2025, with opted-in chats kept for five years; Google tells consumer Gemini users "Please don't enter confidential information that you wouldn't want a reviewer to see" (Google). At the top end the strongest models are the least private: Fable 5.1 requires 30-day retention and is not eligible for zero data retention without express authorisation (Anthropic). The tool-by-tool table is in the confidentiality checklist.

Three habits that follow

  1. Give the model the source; never ask it to remember the law.
  2. Read every citation in the reporter before it leaves your desk.
  3. Know which tier of which vendor you are on before anything client-related goes in.

Module 1 of the AI Lab for Lawyers course covers these fundamentals and the prompting habits that follow from them.

Frequently asked questions

What is a large language model in plain English?

A large language model is a very large statistical model of text. It has read an enormous corpus and learned to predict which token comes next, and from that single ability it can draft, summarise, translate and answer questions. It has no database of law inside it and no separate step that checks whether a citation exists.

Which LLMs do legal AI tools actually use in 2026?

Mostly the same few. Harvey runs Claude Opus 5 and Fable 5.1 alongside OpenAI, Google and Mistral models plus its own Tenet model. Thomson Reuters rebuilt CoCounsel Legal on Anthropic's Claude Agent SDK with its own legal LLM. Lexis+ with Protégé uses Anthropic, Google and OpenAI models. OpenAI's Astra for Law is GPT-6 Astra configured for legal work.

Does an LLM remember what I type into it?

The model's weights do not change when you type, so the model itself does not remember. The service around it may log, retain or train on your input depending on the tier: consumer ChatGPT trains unless you opt out, consumer Gemini chats may be read by human reviewers, and enterprise or API tiers generally do not train by default. Check the tier, not the brand.

Put this into practice

AI Lab for Lawyers: a live, hands-on course for practising lawyers

4 live sessions (8 hours), recordings included, taught by Dr. Niklas Schmidt of Wolf Theiss. Rated 4.7/5 from 381 ratings; alumni from Norton Rose Fulbright, Paul Hastings, Pillsbury and Taylor Wessing.

Reserve a seat · €200 Maven money-back guarantee · certificate of completion