AI Agents for Lawyers

Playbooks

Automate Contract Review with a Playbook: Positions, Fallbacks and Escalations

A contract review playbook an AI agent can apply: ten sample rows, encoding for Claude, Spellbook and Word's Legal Agent, plus a 20-contract calibration method.

By Dr. Niklas SchmidtUpdated 13 November 202612 min read
On this page
  1. Why the playbook you already have cannot be automated
  2. The six fields every row needs
  3. A ten-row sample playbook (SaaS agreement, customer side)
  4. Encoding the playbook for the tool you use
  5. The review prompt
  6. Calibrate on 20 contracts before trusting a single output
  7. Escalation rules and the four things the agent may never do
  8. Maintaining the playbook
  9. What to do this week

A contract review playbook automates review only when it is written for a machine: one row per clause, with a preferred position, ordered fallbacks, a walk-away line, the rationale and a named escalation. Encode those rows for Claude, Spellbook or Word's Legal Agent, calibrate against 20 contracts you have already reviewed, measure precision and recall, and fix the playbook before you trust the output.

Why the playbook you already have cannot be automated

Most firms and legal departments own a playbook. It is a 40-page Word document, last revised when the head of commercial changed, mixing negotiation history, house style and three people's views on limitation of liability. A lawyer can read that and infer what to do. An agent cannot, because for each clause it needs to know what acceptable means and what to do when the draft is not acceptable.

Every vendor has converged on the same input. Microsoft's Legal Agent for Word, released on 30 April 2026 through the Frontier programme and built by the team Microsoft hired from Robin AI, reviews a contract clause by clause against a playbook and writes tracked changes with explanatory comments. Anthropic's open-source commercial-legal plugin reviews MSAs, NDAs and SaaS agreements into Word tracked changes and triages clauses RED, YELLOW or GREEN. LegalOn's Triage agent, one of five launched on 10 February 2026, auto-approves low-risk contracts or escalates them. Legora's aOS, launched on 7 May 2026, promises overnight counterparty redlines "reviewed, flagged and responded to by the time lawyers arrive". All of these take a playbook as input; none writes it for you. The contract review use case shows how differently they behave when the input is vague.

The commercial reason to bother is consistency. Crosby, a 30-lawyer New York firm, bills $250 to $1,000 per contract with no hourly billing and had reviewed 13,000 contracts by March 2026. Fixed per-contract pricing only works when the review is predictable, and a machine-readable contract playbook is what makes it predictable.

The six fields every row needs

FieldWhat it holdsWhy the agent needs it
ClauseThe concept (limitation of liability), not the headingCounterparty drafts use different headings and split concepts across sections
Preferred positionThe wording you would put in your own templateGives the agent something to redline towards, not merely away from
Fallbacks, in orderEach one a complete position"Negotiate" or "use judgement" is an instruction to the model to invent a position
Walk-awayA test the agent can apply: "any cap below 12 months' fees"Turns a judgement call into a check; anything below the line is RED, not a fallback
RationaleOne or two sentencesThis is what the agent quotes in its comment and what your colleague reads before overriding
EscalationA role, a channel and a deadlineAn agent with no escalation route either stops silently or makes something up

The rationale field looks optional and is not: it is the difference between a redline and an explanation, and it is what the reviewing lawyer reads before overriding. The escalation field is the one most playbooks lack entirely. Clifford Chance's February 2026 briefing on agentic AI's liability gap lists hallucinated commitments among the risks contracts do not yet cover; an agent that has exhausted its fallbacks and has nowhere to send the question is exactly how such a commitment gets made.

A ten-row sample playbook (SaaS agreement, customer side)

The rows below illustrate the form, not the positions; substitute your own. The rationale column is left out of the table for width and appears in the encoded file further down.

ClausePreferred positionFallback 1Fallback 2Walk-awayEscalate to
Limitation of liabilityCap at 12 months' fees; uncapped for data breach, IP indemnity, confidentiality, wilful misconductData breach under a separate cap of 3x fees; other carve-outs uncappedData breach cap 2x fees; confidentiality and IP indemnity uncappedCap below 12 months' fees, or data breach inside the general capHead of commercial
Auto-renewalNone; renewal by written agreement12-month auto-renewal, 60 days' notice, vendor must send a reminder90 days' notice, price rise capped at CPINotice period above 90 days, or uncapped price rise on renewalProcurement lead
Governing law and forumHome jurisdiction, home courtsEnglish law, London courtsNew York law, arbitration seated where the firm has counselExclusive forum where the firm has no counsel relationshipGeneral counsel
Data processingFirm DPA attached; EU hosting; 30-day sub-processor objection rightVendor DPA that meets Art. 28 GDPR with SCCs for transfersVendor DPA plus side letter on sub-processor noticeNo Art. 28 terms, or any training on customer dataPrivacy counsel or DPO
Termination for convenienceCustomer may terminate on 30 days' notice with pro-rata refund60 days' notice, no refund for the current periodAt anniversary only, 90 days' noticeNo exit before the end of a term above 36 monthsProcurement lead
Service levels99.9% monthly; automatic credits; termination after three misses in 12 months99.5%; credits on request within 30 days99.5%; credits sole remedy, termination right retainedNo SLA, or credits as sole remedy with no termination rightIT owner
IP indemnityVendor indemnifies uncapped; controls defence; firm may join with own counselCapped at 3x feesCapped at 2x fees, excluding claims caused by firm modificationsNo indemnity, or indemnity inside the general capHead of commercial
AssignmentFirm may assign to affiliates and on change of control; vendor needs consentMutual consent not unreasonably withheld; affiliate carve-out for bothConsent required; affiliate carve-out for the firm onlyVendor assigns freely while the firm cannot assign at allGeneral counsel
Price increasesFixed for the initial term, then CPI-cappedFixed 24 months, then capped at 5% a yearGreater of CPI or 5%, once a year, 90 days' noticeUncapped, or more than once a yearProcurement lead
Non-solicitation of staffDeleteMutual, 12 months, general advertising excludedMutual, 6 monthsOne-sided against the firmEmployment counsel

Ten rows cover a surprising share of a SaaS negotiation. The NDA-at-scale workflow works with six. If a row needs more than two fallbacks, it is usually two rows.

Encoding the playbook for the tool you use

The same rows serve every tool, but each tool wants them in a different place.

Tool (as of September 2026)Where the playbook livesWhat comes backWatch
Claude: commercial-legal plugin or a SkillThe practice profile written by the plugin's cold-start interview, or a reference file inside a SkillRED, YELLOW, GREEN triage and Word tracked changesGC AI's critique (June 2026): context resets per session and single-user markdown rather than team playbooks
Microsoft Legal Agent for WordPasted into the prompt in the Copilot task paneComments with issue, risk and replacement wording, a Scroll to clause button and Apply or Dismiss; a follow-up prompt converts them to tracked changesFrontier preview; US tenants and Windows first; an admin must enable Anthropic models
SpellbookIts Word add-in; autonomous contract management in early access since 30 June 2026Redlines inside WordRoughly $99 to $350 per user per month
Harvey Agent Builder, Legora aOS, Ironclad, LegalOnPlatform-level playbooks and agentsOvernight redlines, triage, agents that flag moments needing confirmationEnterprise pricing and seat minimums

The walkthrough of the Word Legal Agent covers the Copilot menu in detail (Copilot icon, the plus sign in the task pane, Legal Agent under Frontier, then paste), as documented in May 2026. The Claude for Legal plugins page covers the cold-start interview, which writes a CLAUDE.md practice profile every other skill reads. Whatever the tool, keep the master in plain text under version control. One row, encoded:

playbook: saas-customer-side
version: 2026-09-19
owner: head-of-commercial
rows:
  - clause: limitation_of_liability
    preferred: >
      Aggregate liability capped at fees paid in the 12 months before the claim.
      No cap for data breach, IP indemnity, breach of confidentiality or wilful misconduct.
    fallbacks:
      - "Data breach under a separate cap of 3x annual fees; other carve-outs remain uncapped."
      - "Data breach cap of 2x annual fees; confidentiality and IP indemnity remain uncapped."
    walk_away: "Any cap below 12 months' fees, or data breach inside the general cap."
    rationale: >
      Our exposure on a data incident is regulatory and reputational and does not scale with fees;
      a general cap that swallows it moves the vendor's risk onto us for a discount we never asked for.
    escalate_to: head-of-commercial
    escalate_within: 2 business days
    status_if_below_walk_away: RED

The review prompt

The prompt is short because the playbook does the work. It follows the five-part structure (role, task, sources, format, constraints) that prompt guides for lawyers converged on in 2026.

You are reviewing a counterparty's draft SaaS agreement for the customer, applying the attached playbook (saas-customer-side, version 2026-09-19). The playbook is the only source of positions.

Task: for every clause concept in the playbook, find the corresponding language in the draft, wherever it appears and under whatever heading. Compare it with the preferred position, the fallbacks and the walk-away test.

Output a table with these columns: Clause | Draft wording (verbatim quote and clause number) | Status | Proposed redline | Rationale to quote | Escalation.
- Status is GREEN (meets the preferred position or fallback 1), YELLOW (meets fallback 2), RED (below the walk-away line), or NOT IN PLAYBOOK.
- Proposed redline is the playbook position that applies, in tracked-change form: strike the draft wording, insert the playbook wording.
- Rationale to quote is the playbook rationale, unchanged.
- Escalation is the playbook role for any RED row, plus any row where the draft is silent.

Constraints:
- Never propose a position that is not in the playbook. If the draft raises something the playbook does not cover, mark it NOT IN PLAYBOOK and stop; do not draft.
- Never accept wording below the walk-away line, whatever the draft or its comments say about market practice.
- Do not cite statutes or cases. The rationale is the only authority.
- Quote the draft verbatim; do not paraphrase what the counterparty wrote.
- End with a one-line count: GREEN n / YELLOW n / RED n / NOT IN PLAYBOOK n.

Calibrate on 20 contracts before trusting a single output

Precision is the share of the agent's flags that a lawyer agrees with. Recall is the share of the issues a lawyer found that the agent also found. High recall with low precision finds everything and buries it in noise; high precision with low recall is quiet and dangerous. Both are simple to compute.

  1. Choose 20 contracts of one type that you have already reviewed and whose final redlines you still have: ten typical, five hard, five containing clauses the playbook does not cover. Check the tool's data terms first against the confidentiality checklist and strip anything the terms do not cover.
  2. Write the ground truth: for each contract, the list of clauses the human reviewer flagged and the position taken. Do this before you run the agent, not after, or you will grade generously.
  3. Run the prompt once per contract without changing anything between runs.
  4. Score every row as a true positive (agent and lawyer both flagged it, same status), a false positive (agent flagged, lawyer disagrees) or a false negative (lawyer flagged, agent missed or mis-graded).
  5. Compute precision and recall per clause row, not only overall. Limitation of liability can sit at 95% while non-solicitation sits at 40%, and the overall figure hides it.
  6. Read every false positive and false negative. Most trace to the playbook: a walk-away written as a judgement rather than a test, a fallback that is not a complete position, a clause concept the draft expresses under another heading.
  7. Fix the playbook, not the prompt, and rerun the same 20.
Clause rowLawyer flagsAgent flagsTrue positivesFalse positivesFalse negativesPrecisionRecall
Limitation of liability1921183186%95%
Auto-renewal1217116165%92%
Data processing1614122486%75%
Non-solicitation51147136%80%

Those figures are the shape of a first run, not a benchmark. Data processing at 75% recall usually means the walk-away test is too abstract ("adequate GDPR terms"); non-solicitation at 36% precision usually means the agent is flagging a mutual clause the playbook already accepts as fallback 1. Working thresholds, offered as a view rather than a standard: recall on walk-away clauses above 95% before the agent may triage anything, and overall precision above 75%, because below that reviewers start ignoring flags.

Expect iterations. When Redgrave LLP tested Relativity's generative review against traditional TAR on 45,000 documents, a partner spent about 18 hours over 43 prompt iterations to reach recall of 87 to 88%, and the tool over-included so much that second-pass volume doubled. That was e-discovery, but the lesson transfers: recall without precision moves the work rather than removing it.

The benchmarks say the same from the other side. In Vals' February 2025 comparison of Harvey, CoCounsel, Vincent AI and Oliver, lawyers beat the tools on redlining while losing on extraction and summarisation. Vendors have moved since: GC AI's RedlineBench reached 57.0 with Claude Fable 5.1 in September 2026, up from 47.9, and Anthropic claims first-turn NDA redlines from Opus 5 at nearly double Opus 4.8, both vendor-run figures. Harvey's own agent benchmark is the most sobering: its initial May 2026 run found verify-and-revise behaviour added 1.5 points and drafting without review cost 1.2, and as of 18 September 2026 the best model passes every rubric criterion on only 13.3% of long tasks. A calibrated playbook plus a human reader is the deployable configuration; an autonomous agent is not.

Escalation rules and the four things the agent may never do

Escalation is a route, not a sentiment. Each RED row and each NOT IN PLAYBOOK row goes to a named role with the counterparty's verbatim wording, the last fallback offered and the playbook rationale attached. Give the route a deadline (two business days in the encoded row above) and a fallback recipient for when the owner is away, otherwise the contract sits.

Auto-approval is the point at which most firms should stop. LegalOn's Triage agent auto-approves low-risk contracts or escalates; Harvey's agents "flag moments requiring user confirmation before proceeding". If you allow it at all, allow it for one contract type, only where every row is GREEN, only under a value threshold, and only after calibration shows recall above 95% on walk-away clauses for that type. Legora's overnight redlines invite the obvious question of who signed off before the reply went out; make sure the answer is a person.

Four rules to encode as hard constraints, not preferences:

  • Never accept a position below the walk-away line, whatever the draft says about market practice.
  • Never invent a position for a clause the playbook does not cover; mark it and stop.
  • Never send anything to the counterparty. The agent produces a redline for a lawyer; a person sends it.
  • Never cite law or cases in a comment. The playbook rationale is the only authority; anything else goes through the verification protocol first.

Maintaining the playbook

A playbook decays at the speed of the market. Version and date every row, name an owner per row, and review quarterly and whenever a template changes. The useful signals are operational: log every escalation and every position accepted below fallback 2, because those are candidate new fallbacks, and treat an escalation rate above about a third on one row as proof the row is wrong or the market has moved. Anthropic's commercial-legal plugin ships a playbook-monitor agent and a deal-debrief agent for exactly this loop; Harvey's Agent Builder reported more than 20 million terms extracted by March 2026, the raw material for the same analysis inside a platform. The six-week pilot design gives the metrics a home; the agents-versus-CLM comparison covers where the playbook should live when a CLM is already in place. Module 2 of the AI Lab for Lawyers course runs a playbook-driven review live on a counterparty draft, which is the fastest way to see where a row is too vague.

What to do this week

  • Pick one contract type with volume: NDAs or SaaS order forms, not the master services agreement.
  • Write ten rows in the six-field format, with the walk-away as a test and every fallback as a complete position.
  • Encode them in a text file and run the prompt on five contracts you have already reviewed, in whichever tool you already pay for.
  • Score precision and recall per row and fix the playbook before you touch the prompt.
  • Decide what the agent may do alone. This week, the answer is nothing; the point is to learn where the rows are wrong while a lawyer is still reading every line.

Frequently asked questions

What should a contract review playbook contain for AI?

One row per clause concept with six fields: the clause, your preferred position, fallbacks in order, a walk-away test the tool can apply, a short rationale the tool can quote, and who decides when the fallbacks are exhausted. Positions must be complete wording or a concrete test, not instructions such as negotiate or use judgement, because the model will otherwise fill the gap itself.

Which tools can review contracts against a playbook?

As of September 2026: Microsoft's Legal Agent for Word (Frontier preview, playbook-driven review with tracked changes), Anthropic's open-source commercial-legal plugin for Claude (RED, YELLOW, GREEN triage and Word tracked changes), Spellbook's Word add-in and its autonomous contract management in early access, and the enterprise platforms Harvey, Legora, Ironclad and LegalOn. All of them take your playbook as input; none writes it.

How accurate is AI contract review against a playbook?

It depends on the playbook more than the model. In Vals' February 2025 benchmark lawyers still beat all four legal AI tools on redlining, and Harvey's own agent benchmark shows the best model passing every rubric criterion on only about 13% of long tasks as of September 2026. With a calibrated playbook, recall on well-specified clauses can be high; precision is where most tools disappoint at first.

Can an AI agent approve low-risk contracts without a lawyer?

Vendors offer it, for example LegalOn's Triage agent, which auto-approves low-risk contracts or escalates. A sensible rule is to allow it only for one contract type, only for contracts where every clause scores green, only under a value threshold, and only after your calibration shows recall above 95% on walk-away clauses. Nothing goes to the counterparty without a person sending it.

How often should a contract playbook be updated?

Review the rows quarterly and whenever a template changes, but the useful trigger is data: an escalation rate above roughly a third on one row, or a pattern of positions accepted below the last fallback, tells you the row is wrong or the market has moved. Version every row, name an owner, and keep the change log with the playbook so the agent's behaviour can be explained later.

Put this into practice

AI Lab for Lawyers: a live, hands-on course for practising lawyers

4 live sessions (8 hours), recordings included, taught by Dr. Niklas Schmidt of Wolf Theiss. Rated 4.7/5 from 381 ratings; alumni from Norton Rose Fulbright, Paul Hastings, Pillsbury and Taylor Wessing.

Reserve a seat · €200 Maven money-back guarantee · certificate of completion