Begin with the operation, not the agent count
Some work follows a fixed rule: count files, validate required columns, or reject a duplicate identifier. Other work requires interpretation: decide whether a passage describes a proposal or an accepted change. Still other work involves an authorized decision or external action. Separating these operations helps you decide which component should do each job.
Routing directs inputs to different follow-up tasks. Anthropic presents it as useful where categories are distinct and classification can be done accurately. It is an architectural option, not a universal requirement to use multiple models. [source]
Design a small document pipeline
The following is a proposed design for a fictional inventory. Begin with copies and no ability to change originals.
| Stage | Component | Observable result |
|---|---|---|
| Enumerate inputs | Script | Stable file list and processing status |
| Extract candidate fields | Model where interpretation is needed | Date, type, source passage, uncertainty |
| Validate output | Code | Required fields, valid IDs, allowed values |
| Resolve ambiguous cases | Human or further analysis | Recorded disposition and supporting evidence |
| Approve any later rename | Authorized reviewer and separate operation | Exact approved mapping and reversal record |
Passing a structural check means the output has the required shape. It does not establish that the extracted date is correct. Model-based grading also has limitations and should be compared with human judgments for the task. [source]
Make consequential boundaries enforceable
If an agent should only propose names, give it reading access and a separate output destination. Do not rely on “never rename files” while exposing unrestricted file operations. A script can reject collisions before applying an approved mapping. A budget limit can stop further calls regardless of whether the model notices that it has exceeded the intended effort.
Keeping document text separate from instructions is helpful, but text labels do not create a security boundary. Limit available actions and inspect untrusted inputs as part of a layered design. [source]
Test the router as well as the specialist
Using a cheaper model for initial classification may reduce inference cost. It can also hide relevant documents before a stronger model sees them. Include ambiguous examples and inspect a sample of items routed away from further review. Measure missed important cases, false alarms, escalation volume, and total review time.
For a first trial, let uncertain items proceed to review rather than discarding them. A self-reported confidence of 95% is not proof that a classification is correct. Any numerical threshold requires evidence about performance on representative labelled cases.
Try it: cheap routing misses the exception
A classifier routes an email as routine because its subject is “Schedule.” Buried in the text is a disputed change to the delivery condition. The downstream reviewer never receives it. Where did the workflow fail?
Check your understanding
The routing stage excluded relevant material. Improve and test routing coverage; improving the downstream prompt alone cannot recover an unseen document. Consider whether routing is worthwhile at this scale. Compare the simpler baseline that reviews every item. Retain the more complex pipeline only if its measured benefits justify the additional failure points and maintenance.