A ministry spent nine months checking. You spent an evening

Short answer. The UAE Ministry of Justice cut pre-trial case review from 18 working days to two hours using six agents. Before that the model was calibrated for nine months, trained on more than 22,000 legal documents and tested against 10,000 archived case files where the right answer was already known. What transfers to a company of twenty is the order of work, not the budget: archive first, live data second.

Every agent demo works. The question is what happens on the case that does not look like the demo, and there is a cheap way to find out.

The ministry put six agents on case files and kept the judge

What it did before that is the part worth copying.

On 27 September 2026 the UAE Ministry of Justice presented the system at the 15th UN Crime Congress. According to the ministry, pre-trial case analysis and evidentiary report summarisation drop from 18 working days to two hours.

The platform runs six specialised autonomous agents. They work through case dockets, cross-reference submissions and turn the evidence into structured briefs for the presiding judge.

The number that matters sits before the launch. The underlying model was developed and calibrated over nine months, trained on more than 22,000 legal documents, and tested across 10,000 historical case files, in compliance with the UAE's federal personal data law.

The ministry was also explicit about the limit. In its words, the AI "assists by synthesizing records and suggesting structured judgment templates, but the judge retains absolute authority to evaluate, modify, rewrite, or dismiss any automated output". Rollout to the federal courts is planned for later this year, after trials conclude.

Why is a run over the archive the actual test?

Because in the archive you already know what the answer was.

A demo shows you three cases, and the three were chosen. A pilot on live clients shows you whatever walked in this week, and you find out how it went when somebody complains. An archive gives you a third thing: a set of cases with outcomes already recorded, which turns "looks plausible" into "matched or did not".

The trap is that an agent's output reads as competent in both states, because the same machinery produces fluency and accuracy. Without a known answer to compare against, you are grading the writing.

The archive also carries the cases a demo never contains. The client who changed their mind twice, the invoice with the wrong currency, the enquiry that arrived in three languages. Those are rare enough to miss in a sample of three and common enough to appear within a month of launch.

We do not know how the ministry counted agreement, and the report does not say. The point stands regardless of their method: they had a body of decided cases and they used it before going near a live one.

What archive does a twenty-person company have?

The same kind everybody has: closed deals, answered threads and last year's reports.

A brokerage has the enquiries that became viewings and the ones that went quiet. An accounting practice keeps the filings it already submitted, with the corrections that followed. A corporate services firm has the applications that cleared and the ones the authority sent back.

Three rules make that archive useful.

1. Take 30 to 50 cases, not 5. Below thirty, one unusual case moves the result enough to mislead you in either direction. Above fifty, you are spending a week on something that should cost a day.
2. Include the ones that ended badly. The refunds, the complaints, and the deals that collapsed at the last step. An archive of successes tells you the agent handles success.
3. Hide the outcome while it runs. Give the agent what the person had at the time, and keep the resolution out of the prompt. Feeding it the answer and then admiring the answer is the most common way this test gets faked.

If the archive is not digitised, the test is still possible. Take the last thirty files from wherever they live, in whatever form, and work through them by hand once. Doing it manually is slower than a script and still faster than finding out from a client.

How much authority the agent gets after it passes is a separate design question, and we covered it in the piece on agents that advise while people decide.

What does an honest launch sequence look like?

Four steps, and live data only appears at the third.

1. Pick one process and one measurable outcome. Not "improve support" but "classify the enquiry into the right category". You cannot score an outcome you have not named.
2. Run the archive with the outcome hidden. Record every disagreement between the agent and the record. The value is in the disagreements themselves, and a single percentage averages them away. Half of them usually turn out to be the archive being wrong, which is its own finding.
3. Run in parallel on live work. The agent does the task, a person does it too, and nothing the agent produces reaches a client. Two weeks of this costs little and catches what the archive could not.
4. Hand over the routine share only. The cases it agreed on go to the agent, the rest stay with the person, and the boundary is written down rather than understood.

The stopping rule belongs in step two. Decide, before you look, what agreement rate sends the agent back for rework, and write the number down first. Deciding it afterwards is how a 70% result becomes "quite promising".

The short version

• The interesting number in the ministry's announcement is nine months of calibration, not two hours of runtime.
• An archive with known outcomes is the only cheap way to tell competence from fluency. Every company has one.
• Cases that ended badly are the ones worth testing on. They are also the ones nobody puts in a demo.
• Set the pass mark before the run. A threshold chosen after seeing the result is not a threshold.

We build agents for real processes and we do this run before anything touches a client, because the alternative is finding out from the client. How that works is on the AI team page. You can start before you call anyone: pull your last thirty closed cases and decide what a correct answer would have looked like on each.