Use rules when reliable input fields determine the route, a trained classifier when stable categories recur in enough labeled examples, and an LLM when interpreting the request requires language and context that simpler methods do not handle well. Every approach needs a route for ambiguity and a separate authorization check before consequential actions.
Routing decides which workflow should examine a task. It should not silently decide that the task is approved. “This looks like a billing request” and “this user may issue a refund” are different conclusions.
Define routes by the work that follows
Consider a fictional developer-support queue with three destinations: documentation help, defect investigation, and account changes. The last destination prepares a proposal for an authorized operator; it does not execute account changes automatically.
Write the contract for each destination before choosing a classifier. Documentation help explains existing behavior. Defect investigation needs a reported mismatch and supporting details. Account changes need the target account and the requested effect. Mixed or underspecified requests can enter a clarification queue.
Anthropic's routing pattern describes classifying an input and sending it to a specialized workflow. It allows either an LLM or a more traditional classification approach. The pattern comes from a 2024 engineering article; it supplies architectural context, not a current ranking of routing products.
If two routes accept the same tasks and differ only in their names, the routing problem may be unnecessary. Clarify the operational difference before adding a model to resolve a taxonomy nobody can explain.
Compare three approaches on the same requests
The following table is a design analysis of a small synthetic collection. It records expected difficulties, not predictions observed from trained models.
| Request | Intended treatment | What the routing design must handle |
|---|---|---|
| “Where is the export retention documentation?” | Documentation | A straightforward informational request |
| “Export returns HTTP 500 for this sample.” | Defect investigation | An observed failure with a reproduction lead |
| “Remove my teammate's access to workspace A.” | Account-change review | An effect requiring identity and authority checks |
| “The export guide says 30 days, but my trial archive vanished after 7.” | Clarify or investigate the applicable policy | Possible documented exception, not automatically a defect |
| “Explain access roles, then remove Lee.” | Split explanation from proposed change | Two different downstream workflows |
| “Fix it.” | Clarification | Missing object and intended behavior |
A keyword rule routing every mention of “access” to account changes would mishandle informational questions about roles. A rule based on a validated form field such as request_type=access_change is stronger, provided the field is trustworthy and matches the task.
A classifier can learn recurring wording from labeled examples, but label quality and coverage become part of its contract. An LLM can interpret a longer request and propose decomposition, but it can still choose a plausible wrong route or produce an invalid label. The table does not establish which would make fewer mistakes.
Keep deterministic gates where they belong
Some decisions should precede classification: whether the user is authenticated, whether the supplied workspace is accessible, whether a required identifier is valid, and whether the operation is supported. The model should not guess these facts from prose.
After classification, validate the route against the allowed set and pass only the context and capabilities needed by that workflow. An account-change route can prepare a reviewable request using read access. It does not need unrestricted account administration merely to classify the request.
A structured routing output could be:
{
"route": "clarification",
"reason_code": "mixed_intent",
"evidence": "Explain access roles, then remove Lee.",
"proposed_parts": ["explain_roles", "review_access_removal"]
}The evidence is a short input excerpt and the reason is an application code, not a request for hidden reasoning. A downstream validator can check that the route and proposed parts are supported. It must still reject an unauthorized effect.
Treat abstention as an operating choice
A forced choice hides uncertainty by sending every input somewhere. An explicit clarification route makes uncertainty observable, but it consumes user or operator time. Choose that tradeoff according to the cost of a wrong route.
For a probabilistic classifier, the decision threshold is separate from its estimated scores. The scikit-learn threshold guide explains that default cutoffs need not fit a use case and discusses tuning against a selected objective. Its binary-classification examples do not directly prescribe a threshold for a multiclass router with abstention.
Likewise, an LLM saying “confidence: 0.97” is not automatically a calibrated probability. Evaluate the routing decision against labeled cases. If scores are used to choose abstention, verify their relationship with actual errors on held-out data.
Evaluate the router and the destination separately
Start with requests labeled by people who understand the downstream workflows. Include ambiguous requests, multiple intents, unfamiliar phrasing, unsupported languages if relevant, and examples outside the intended domain. Keep a held-out set separate from prompt and rule development.
Record a confusion matrix with actual and predicted routes, including clarification. The scikit-learn evaluation reference defines that matrix as counts for each true/predicted class pair. Inspect the errors themselves: sending a documentation question to a slower help queue differs from sending an unauthorized change toward an executor.
Track coverage, abstention, correction effort, and time or usage when actually measured. Then measure whether correctly routed tasks succeed downstream. A perfect router cannot repair a defect-investigation workflow that never checks the reproduction.
Choosing an agent or script addresses the wider amount of discretion in automation. Routing is a smaller decision that can combine explicit gates, a bounded interpretation step, and deterministic validation.
Begin with the simplest route policy your evidence supports. Add a classifier or LLM when a defined class of routing failures justifies it, and keep the same held-out cases so the added complexity must demonstrate a useful change.
Documentation reviewed October 28, 2026. The requests and expected treatments are conceptual. No classifier was trained, no LLM was called, and no comparative accuracy or cost was measured.
