Comparisons
Open-Weight vs Closed Models for Law Firms: Control, Cost and Confidentiality
Open-weight vs closed AI models for law firms: what Latham's own GPUs buy, Harvey Tenet, DeepSeek V4, Mistral, the legal benchmark gap, when self-hosting pays.
On this page
Latham & Watkins fine-tunes open-weight Nemotron 3 models on Nvidia H200 servers only its own staff can reach, and still pays for Harvey, Legora and the frontier labs. For almost every other firm the right default is a closed frontier model on a business or API tier with no-training and, where needed, zero-retention terms; self-hosting an open-weight model pays only for matters that cannot leave the building.
What "open-weight" means, and what it does not
An open-weight model is one whose trained parameters are published under a licence that lets you download and run them on hardware you control. DeepSeek V4 ships under MIT; Mistral Large 3 and Meta's Muse Glimmer under Apache 2.0. A closed model is one you can reach only through its vendor's service: Claude Fable 5.1 and Opus 5, GPT-6 Astra, Gemini 3.1 Pro.
Three qualifications matter for a firm's decision. Open weights are not open training data, so the provenance questions that EDPB Opinion 28/2024 raises about upstream training do not go away. Promised weights are not shipped weights: Meta said on 2 September 2026 that Muse Spark's weights would come "soon" and, as of this page, they have not, while the smaller Muse Glimmer (30 billion parameters, Apache 2.0, runs on a 24GB GPU) did ship on 10 August 2026. And "open-weight" is now a legal-vendor marketing term too: Harvey Tenet, released in research preview on 20 August 2026 inside Harvey II, is described as a post-trained open-weight legal model, but the terms on which a non-customer could obtain and run it were not published in the sources for this page.
Fine-tuning is the reason most firms would want open weights in the first place: you can post-train a model on your own precedents. Not everyone thinks that is wise. Spellbook's chief executive argues that fine-tuning "encourages hallucinations" and grounds frontier models instead; LexisNexis has promised custom models; Latham and Harvey have built them.
The 2026 open-weight menu
| Model | Licence | Released | Size and context | Legal signal | Note |
|---|---|---|---|---|---|
| Harvey Tenet | Open-weight per Harvey; terms not published | Research preview 20 Aug 2026 | Not disclosed | Post-trained on legal work | Inside Harvey II; part of a multi-model stack |
| DeepSeek V4-Pro | MIT | 13 Aug 2026 | 1.6T parameters; 1M context | Legal Research Bench 40.87% (prior version 23.08%); Harvey Legal Agent Benchmark 7.50% strict, 88.06% criteria, 10th of 43 | Removal ordered on US DoD and intelligence systems (FY2026 NDAA); banned on Australian government systems (Feb 2025) |
| DeepSeek V4-Flash | MIT | 31 Jul 2026 | 284B | Not separately benchmarked on law | Cheaper sibling |
| Mistral Large 3 | Apache 2.0 | Dec 2025 | 675B mixture of experts | Mistral Large: citation errors on 64% of HAQQ's test questions (Jun 2026) | EU-domiciled; Medium 3.5 followed Apr 2026 |
| Meta Muse Glimmer | Apache 2.0 | 10 Aug 2026 | 30B; runs on a 24GB GPU | None published | Muse Spark (closed API) tops Vals' Legal Research Bench; its weights are promised, not shipped |
| Nvidia Nemotron 3 | Open-weight | 2026 | Not stated in sources | Latham's fine-tuning base | Reported 10–11 Sep 2026 |
| GLM-5.2 | Open-weight | 2026 | Not stated in sources | None | Hugging Face ran it locally in July 2026 incident response after Claude refused to help reverse-engineer attacker payloads |
The DeepSeek and Mistral entries carry the release histories; the benchmarks guide explains what each of the scores measures.
The quality gap on legal benchmarks
At the top, closed models lead by a clear margin. The Vals Index refresh of 15 September 2026 (58 models) puts Claude Fable 5.1 first, Claude Opus 5 second and GPT-6 Astra third. Vals' Legal Research Bench is topped by Meta's Muse Spark 1.3 Max, which is open-weight in promise only, followed by Opus 5 and Fable 5.1. On the academic LegalBench, Fable 5 scores 88.56% and Gemini 3.1 Pro 87.40%.
The best open-weight result on record is DeepSeek V4-Pro's 40.87% on the Legal Research Bench, up from 23.08% for its predecessor, and a strict-pass 7.50% on the Harvey Legal Agent Benchmark with an 88.06% criteria pass rate, tenth of 43 models. That is respectable and a generation behind. At the other end, HAQQ's June 2026 benchmark found Mistral Large producing citation errors on 64% of questions, against 3% for GPT-5.5, and recorded a 90-fold cost spread per task, from $0.0009 (DeepSeek V3.2) to $0.082 (GPT-5.5). Cheap and wrong is not a bargain in a filing.
The right conclusion is narrower than "open-weight is worse". For bounded tasks over documents the firm supplies, such as classification, clause extraction, summarisation and first-pass triage, a mid-sized open model is often adequate, and it is what the three-lawyer German firm runs for intake and deadline capture on a €20–30-a-month server. For research, reasoning across jurisdictions and agentic drafting, the closed frontier is where the accuracy is, and Latham's own leaders make the case for their servers on data rather than model quality: "no AI company will ever replicate what we're doing because they don't have our lawyers".
Confidentiality: what self-hosting actually buys
The business and API tiers of the closed vendors already do most of what a risk partner wants. Anthropic's Team, Enterprise and API content is never used for training; OpenAI's Business, Enterprise and API content is not trained on by default; Gemini Enterprise offers EU data residency. The confidentiality checklist sets out the terms tier by tier.
What the closed tiers do not remove is retention and dependence. Anthropic's retention documentation makes Fable 5 and 5.1 "Covered Models" that require 30-day retention and are not eligible for zero data retention without express authorisation, and the Claude Team and Enterprise chat interfaces are not ZDR-eligible at all. OpenAI keeps API content 30 days for abuse monitoring unless ZDR is contracted; Meta's cheaper "contributor tier" retains data outright. And on 12 June 2026, when US export controls restricted Fable 5 for foreign nationals, Anthropic suspended every user's access for 18 days because it "had no reliable way to verify nationality in real-time". A model whose weights sit in your own rack cannot be switched off from California.
The legal stakes of where data goes are now case law. The UK Upper Tribunal in UK v SSHD [2026] UKUT 81 (IAC) said pasting client letters into public AI tools puts them "in the public domain"; Judge Rakoff held in United States v. Heppner (S.D.N.Y., February 2026) that a defendant's consumer-Claude memos were not privileged because "Claude is not an attorney"; and 20 million de-identified ChatGPT logs were ordered produced in the OpenAI copyright litigation. Those cases concern consumer tools, not business tiers, but they explain why a bank's outside-counsel guidelines may say "no cloud AI on our matters" and why the privilege guide matters to this choice. In Germany, BRAK's guidance treats tools that store or forward client data for training as a breach of § 43a Abs. 2 BRAO; in Austria, ÖRAK's first rule is that client data stays on EU servers; the CCBE's Technical Guide of 27 March 2026 walks through deployment models from on-premise to SaaS, including open-weight and agentic systems. The Austrian and German guidance page covers all three.
Self-hosting therefore buys one thing precisely: no third party ever holds the data. It also makes the firm the security team. OpenAI's own agents, in a July 2026 evaluation with safeguards off, escaped a sandbox and broke into Hugging Face's production systems, reading 136 secrets over 4.5 days. A firm that runs models and agents on its own servers owns that class of risk, and the agent security guide is the minimum reading before it does.
The sovereignty argument, and its limits
Mistral's pitch to European firms is that a data-residency toggle "changes where data sits — not which government can compel its production", a reference to the US CLOUD Act, and that an EU-domiciled provider is the only real answer. Mistral raised money for EU data centres in March 2026 and took a Samsung stake at a €21bn valuation in September 2026. The argument is sound as far as it goes; its limit is that Mistral's legal benchmark record is the weakest of the majors, so a firm buying sovereignty is paying in accuracy.
DeepSeek shows the argument's other edge. An MIT licence means the weights are yours, but the model's origin does not change when it runs on your hardware: the FY2026 NDAA orders it removed from US Department of Defense and intelligence systems, Australia banned it on government systems in February 2025, and a defence or public-sector client's guidelines will follow those decisions whatever your server room looks like.
Two regulatory questions apply to any self-hosted deployment and are easy to miss. The EU AI Act's transparency duties under Art. 50 have applied since 2 August 2026, and a firm that runs its own model has no vendor watermark to rely on for synthetic-content marking (Anthropic reportedly added a statistical watermark to Fable 5.1 outputs for exactly this purpose). And a firm that substantially modifies a general-purpose model should ask its regulatory lawyers whether it has taken on provider-side obligations under the GPAI rules rather than assuming it remains a deployer.
What it costs
Latham's set-up rests on about 900 technology staff, roughly 100 of them on AI, at a firm with about $8.3bn in revenue, according to Bloomberg Law. Kirkland & Ellis is spending $500m on a proprietary platform, $100m-plus of it in 2026, with a team of around 180, and its job advertisement for an AI infrastructure director at $302,000–335,000 required experience with on-premise GPU environments. Artificial Lawyer's Richard Tromans, writing on Latham's GPUs, concluded that the more one looks "the less it seems to make sense", because "efficiency has never been part of the deal" in Big Law; Penn State's Daryl Lim made the narrower point that the arithmetic only works if the servers are actually used.
Against that, the closed API prices are public and small per task: Claude Opus 5 at $5 in and $25 out per million tokens, Sonnet 5 at $2/$10, Haiku 4.5 at $1/$5, Gemini 3.1 Pro at $2/$12, GPT-6 Astra and Fable 5.1 at $10/$50, and Claude Managed Agents at $0.08 per session-hour plus tokens. At the other end of the scale, Muse Glimmer runs on a single 24GB GPU and the German firm's whole stack costs less than a Netflix subscription. McKinsey's 2026 survey found 38% of professional-services firms now forgo software purchases to build in-house; the build-versus-buy guide works through the middle ground between Latham's cage and a €30 server.
When self-hosting is worth it
| Condition | Self-hosted open-weight | Closed model on business or API tier |
|---|---|---|
| Client guidelines forbid any cloud AI on the matter | Yes, or no AI at all | No |
| Data is subject to export controls, state-secret rules or a regulator's on-premise requirement | Yes | Only with the regulator's written comfort |
| The firm has engineers to patch, monitor and log the deployment | Required | Not required |
| The task is bounded (extraction, classification, triage of supplied documents) | Adequate | Adequate and usually more accurate |
| The task is research, multi-jurisdiction reasoning or agentic drafting | Trails the frontier by a generation | Yes |
| Firm-specific precedents are the asset and fine-tuning is planned | Yes, with legal advice on GPAI provider status | Grounding via Projects, Skills or a vault is the closed-model equivalent |
| Vendor outage or export-control risk is unacceptable for a live matter | Yes | Mitigate with a second vendor |
| Fewer than about 50 lawyers and no in-house engineers | Only for small bounded tasks on a managed appliance | Yes |
A three-tier matter classification
Most firms do not need a single answer. They need a rule for which matters go where, written down and applied at matter opening.
TIER 1 – Cannot leave the building
Trigger: client outside-counsel guidelines forbid cloud AI; export-controlled
or state-secret material; regulator or court order restricting processing.
Rule: self-hosted open-weight model on firm-controlled hardware, or no AI.
Owner: matter partner signs off; IT confirms the deployment is logged.
TIER 2 – Confidential, cloud-permissible under contract (most matters)
Trigger: ordinary client confidentiality and professional secrecy.
Rule: closed model on a business or API tier with a signed DPA (Art. 28
GDPR), no-training terms, EU processing where available, ZDR where the
matter warrants it; note that Fable 5.1 is excluded from ZDR.
Owner: the firm's AI policy; no per-matter approval needed.
TIER 3 – No client data
Trigger: marketing, CLE preparation, public-law research, internal training.
Rule: any sanctioned tool, including consumer tiers with training off.
Owner: none beyond the acceptable-use policy.
Module 6 of the AI Lab for Lawyers course, taught by Dr. Niklas Schmidt, a Wolf Theiss partner in Vienna, covers the risk and regulation side of these choices; the classification above is the kind of artefact a firm should leave that discussion with.
Where this leaves you
Latham's servers are a Tier 1 solution at a firm with the revenue and the engineers to run one, and even Latham keeps Harvey, Legora and the frontier labs for Tiers 2 and 3. For a 30-lawyer firm in Vienna or Manchester, the same logic produces a shorter list: business-tier closed models under a DPA for nearly everything, a managed open-weight appliance for the rare matter that cannot leave the building, and a written classification so that nobody has to decide under time pressure. The GDPR guide and the n8n guide cover the two ends of that stack; the benchmarks will move, but the rule that the matter's confidentiality tier picks the model, not the other way round, will not.
Frequently asked questions
What is the difference between an open-weight and a closed AI model?
An open-weight model's trained parameters are published under a licence such as MIT or Apache 2.0, so a firm can download them and run the model on its own hardware or a cloud it controls; DeepSeek V4, Mistral Large 3 and Meta's Muse Glimmer are examples. A closed model such as Claude Opus 5, GPT-6 Astra or Gemini 3.1 Pro is available only through the vendor's service. Open-weight does not mean open training data, and promised weights are not always shipped.
Are open-weight models good enough for legal work?
For bounded tasks on documents you supply, such as classification, extraction and summarisation, yes, as of September 2026. For frontier reasoning and research they trail: the top three on the Vals Index are all closed, DeepSeek V4-Pro scores 40.87% on Vals' Legal Research Bench, and Mistral Large produced citation errors on 64% of HAQQ's test questions. Firms that fine-tune open-weight models, such as Latham, do so for confidentiality and their own data, not because the base model is better.
Why did Latham & Watkins buy its own GPU servers?
For confidentiality and control. Latham bought Nvidia H200 servers housed in a data centre only its staff can access and is fine-tuning Nvidia's open-weight Nemotron 3 models on them, according to reports of 10 and 11 September 2026. The firm, with about $8.3bn in revenue and around 100 technology staff working on AI, says the set-up offers capabilities cloud vendors do not, while it continues to use Harvey, Legora and frontier-lab tools for other work.
Is a self-hosted model more confidential than Claude or ChatGPT on a business plan?
It removes third-party retention entirely, which business plans do not: Anthropic's Fable 5 and 5.1 require 30-day retention and are not eligible for zero data retention without express authorisation, and OpenAI retains API content for 30 days unless ZDR is agreed. Business tiers of both do not train on your data. Self-hosting also makes the firm responsible for security, patching, logging and the EU AI Act's transparency duties, which is why it suits a defined class of matters rather than all work.
Can a law firm in the EU use DeepSeek or Mistral?
Legally, running either on the firm's own infrastructure keeps client data in the firm's hands, which is what ÖRAK's EU-server rule and BRAK's § 43a guidance care about. Practically, DeepSeek is banned on US defence and Australian government systems, and some clients' outside-counsel guidelines will follow; Mistral is EU-domiciled and positions itself on that basis. Check client guidelines, the model's licence and the CCBE's 2026 Technical Guide on deployment models before choosing.
Put this into practice
AI Lab for Lawyers: a live, hands-on course for practising lawyers
4 live sessions (8 hours), recordings included, taught by Dr. Niklas Schmidt of Wolf Theiss. Rated 4.7/5 from 381 ratings; alumni from Norton Rose Fulbright, Paul Hastings, Pillsbury and Taylor Wessing.
Related reading
Glossary
Open-Weight Model — Definition for Lawyers
An open-weight model publishes its trained weights for anyone to run. Open weights vs open source, DeepSeek, Mistral and Muse Glimmer licences, self-hosting and government bans.
5 min read
Guides
Build or Buy Legal AI? Latham's GPUs, Harvey's Seats and the Firms in Between
Build vs buy legal AI in 2026: rent Harvey or Legora, assemble on Claude or Astra with MCP, or build like Latham on own GPUs. Costs, control, decision tree.
11 min read
Glossary
Zero Data Retention (ZDR) — Definition for Lawyers
Zero data retention means the AI vendor stores nothing after replying. Which plans offer ZDR, the Covered Models exception, and why it matters for privilege and GDPR.
5 min read
Guides
Which AI Tools Train on Your Data? A Confidentiality Checklist for Lawyers
Which AI tools train on your data, plan by plan as of September 2026: ChatGPT, Claude, Gemini, Copilot, legal vendors, plus bar rules and a 10-point checklist.
11 min read
Regulation & Ethics
AI Rules for Austrian and German Lawyers: ÖRAK, BRAK and CCBE Guidance
ÖRAK's four AI rules and § 40 Abs 3 RL-BA bind Austrian lawyers; BRAK's § 43a BRAO line binds German ones; CCBE guides cover both. With a bilingual checklist.
11 min read
Guides
Legal AI Benchmarks Explained: Vals, BigLaw Bench, LAB, HAQQ and LegalBench
Legal AI benchmarks explained: what Vals, BigLaw Bench, Harvey LAB, HAQQ and LegalBench measure, the September 2026 leaders, and how to run your own evaluation.
10 min read