Evaluation (eval)
Also: evaluations, evals
Testing an AI tool on a set of tasks to see how well it actually does.
The fuller definition
An evaluation tests a model or workflow against defined tasks and criteria to measure specific aspects of performance.
The reference set has ten relevant events. Six are found, four are missed, and two additional returned items are unsupported. This example measures extraction, not every aspect of a legal work product.
A concrete example
A chronology evaluation checks missed events, unsupported additions, wrong dates, and review time.
The distinction that matters
One impressive demonstration or a broad benchmark does not establish reliability on your own record.
Related terms
Benchmark
A standard test used to compare models. A high score on it does not mean the model is good at your task.
Reliability & securityPrecision
Of the things the system returned, how many were right.
Reliability & securityRecall
Of the things it should have found, how many it actually found.
Where it is taught
Test an AI workflow on work you can assessSource and further reading
Anthropic · Evaluating AI agentsThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.