Glossary
Token (AI) — Definition for Lawyers
A token is the unit of text a language model reads and writes, usually part of a word. Why limits, prices and speed are counted in tokens, not pages or words.
On this page
A token is the basic unit of text a language model reads and writes. It is usually a short word, part of a longer word, a number or a punctuation mark. Models never see pages or words, only sequences of tokens, which is why context windows, output limits, usage caps and API prices are all measured in tokens rather than in pages.
How text becomes tokens
Before a model reads anything, a tokeniser splits the text into pieces drawn from a fixed vocabulary, then maps each piece to a number. Common words in the training data tend to be single tokens; rarer words are broken into several fragments. The guide to how large language models work explains what the model then does with those numbers: it predicts the next token, one at a time, until the answer is complete.
The split is not intuitive. A worked example, illustrative only, because every vendor's tokeniser splits differently:
| Text | Illustrative split | Why |
|---|---|---|
the contract | the · contract | Two common words, two tokens; the space usually travels with the word |
indemnification | indemn · ification | Long legal word, split into familiar fragments |
Gewährleistungsfrist | Gew · ähr · leist · ungs · frist | German compounds and umlauts cost more fragments |
§ 1295 ABGB | § · 129 · 5 · AB · GB | Symbols, numbers and abbreviations fragment heavily |
[2025] EWHC 1383 (Admin) | Many short fragments | Neutral citations are strings of brackets, digits and capitals |
Two consequences follow. First, the rough English rule of thumb, about three-quarters of a word per token, breaks down for other languages, dense citations and tables. A German share purchase agreement fills a context window faster than an English one of the same page count. Second, a citation is a run of odd fragments with weak statistical associations, which is part of why a model can produce a citation with the right shape and no substance; the entry on hallucination covers the rest of that story.
Why limits and prices are counted in tokens
Every token you send has to be processed, and every token the model writes is generated in sequence. Tokens therefore track the real cost of running a model, and vendors express four different limits in them:
- Context window. The total the model can hold at once: your prompt, attached documents, the conversation so far and its answer.
- Output cap. A separate ceiling on how much the model may write in one turn. A long draft that stops mid-clause has usually hit this cap, not the window.
- Price. API pricing quotes one rate per million input tokens and a higher rate per million output tokens. The legal AI pricing guide sets out current figures.
- Rate and usage limits. Chat subscriptions do not show an invoice; they show a usage limit, which is the same token budget expressed differently.
In a multi-turn conversation the model re-reads the whole context on every turn. A bundle loaded at the start is counted again with each follow-up question, which is why long conversations over large documents slow down and consume limits quickly.
Estimating a matter in tokens
Before loading a bundle, estimate it. A method that takes two minutes:
1. Pick one representative page of the document (body text, not the cover).
2. Paste it into the vendor's token counter or the API's token-counting endpoint,
where the vendor offers one.
3. Multiply by the page count; add 20% for schedules, tables and exhibits.
4. Compare the total with the model's context window, leaving room for
your instructions, the conversation and the answer.
5. If the total exceeds roughly half the window, split the task or use a
tool that retrieves passages instead of loading everything.
Step 5 is the point where retrieval-augmented generation becomes the better design: search the corpus, pass the model only the passages that match, and keep the window for reasoning.
Where this leaves you
You do not need to count tokens for everyday drafting. You do need a feel for them when a task involves a long bundle, a non-English corpus or an automated workflow billed through an API, because that is where a model silently truncates, slows down or costs more than expected. Module 1 of the AI Lab for Lawyers course covers tokens and context windows as part of how language models actually work.
Frequently asked questions
How many tokens is a page of a contract?
There is no fixed figure, because it depends on the language, the tokeniser and how dense the page is. As a rule of thumb, English prose runs at roughly four characters or three-quarters of a word per token. Defined terms, clause numbers, tables and citations push the count up. For a reliable number, paste a representative page into the vendor's token counter and multiply.
Why do AI tools charge per token instead of per document?
Because tokens are what the model actually processes. Every token you send is read, and every token the model writes is generated one at a time, so compute cost tracks token count closely. API pricing therefore quotes separate rates for input and output tokens, and chat subscriptions express the same constraint as usage limits rather than an invoice.
Put this into practice
AI Lab for Lawyers: a live, hands-on course for practising lawyers
4 live sessions (8 hours), recordings included, taught by Dr. Niklas Schmidt of Wolf Theiss. Rated 4.7/5 from 381 ratings; alumni from Norton Rose Fulbright, Paul Hastings, Pillsbury and Taylor Wessing.
Related reading
Glossary
Context Window — Definition for Lawyers
A context window is the text an AI model can hold at once. As of September 2026 it is 1 million tokens on Claude, GPT-6 and Gemini, about 555,000 words.
4 min read
Guides
How Large Language Models Work, Explained for Lawyers
How LLMs work, explained for lawyers: tokens, prediction, training versus inference, context windows, knowledge cutoffs, reasoning modes and the September 2026 model line-up.
8 min read
Glossary
Large Language Model (LLM) — Definition for Lawyers
A large language model is a neural network trained to predict the next token, and it powers every legal AI tool. How LLMs are built, the models of September 2026 and what follows for reliability and confidentiality.
4 min read
Guides
What Legal AI Really Costs in 2026: Harvey, Legora, CoCounsel, Claude and ChatGPT
Legal AI cost as of September 2026, from €39 to $1,200 per seat per month, with Harvey, Legora, CoCounsel, Claude and ChatGPT prices and worked budgets for 5 and 50 lawyers.
10 min read
Use Cases
AI Legal Translation: Accuracy, Confidentiality and Workflow
AI legal translation gives reliable working drafts of contracts and pleadings when a glossary, system prompt and bilingual review surround it. German-English example.
10 min read
Glossary
Retrieval-Augmented Generation (RAG) — Definition for Lawyers
Retrieval-augmented generation (RAG) searches a document set first and drafts from what it finds. Every grounded legal tool uses it; here is why errors still get through.
4 min read