Model Routing Decision
Decide which tier handles a task — including the no-model tier, which is the most valuable rule in the policy.
Runtime state: Applied on every task in my own delivery workflow, including the no-model tier. It is a working practice rather than a deployed service.
- Business objective
- Stop overpaying on trivial work and underpowering hard work at the same time, which is what happens when one model handles everything.
- Trigger
- Any task about to be given to an assistant.
- Inputs
- The task and its class
- Sensitivity classification of the data involved
- Current spend against the tracked budget
- Provider availability
- Outputs
- A routing decision, or a decision to use no model at all
- Recorded spend attributed to the task class
- A fallback route if the primary tier is unavailable
The first question is not which model
It is whether a model at all.
A large share of what I was originally sending to an assistant were factual questions about the system — which service owns this table, what does this flag default to, how many of these exist. A query, a search or a script answers those faster, for free, and exactly correctly. A model answers them approximately correctly, which is worse in every respect.
The no-model tier is the most valuable rule in this policy.
The workflow
Task to routing decision
Two gates before a tier is even considered.
- Approval gate
Sensitivity classification
Decision point: Does this involve secrets or production data? If so, it does not enter a prompt at all — regardless of tier.
- Approval gate
Is this a deterministic fact?
Decision point: Can a query, a search or a script answer this exactly? If yes, route to no model. Faster, free, and not capable of being wrong.
- Automated software
Classify the task
Extraction, summarization, mechanical transformation, architecture, security review, or production incident.
- Automated software
Select the tier
Decision point: Extraction and summarization — inexpensive. Architecture and security — strong. Production incident — premium reasoning, where cost is not a consideration.
- Automated software
Check rate and budget
Decision point: Within limits? If not, defer or downgrade — except for incidents, which are never downgraded.
- AI / LLM
Execute on the selected tier
Under an explicit token, tool-call and runtime ceiling.
- Monitoring / feedback
Record spend against the task class
Attributed spend is what turns the routing policy from a guess into something evidence can correct.
- Automated software
Fallback if the tier is unavailable
Decision point: Defined alternative per tier. Availability problems degrade the tier, never the approval boundary.
- Automated software
- AI / LLM
- Approval gate
- Monitoring / feedback
The tiers
| Tier | Task class | Why |
|---|---|---|
| No model | Deterministic facts, counts, lookups, mechanical checks | Exactly correct, instant, free. A model can only make this worse. |
| Inexpensive | Extraction, summarization, mechanical transformation | The task is well-specified and the output is easy to verify. |
| Strong | Architecture, security review, non-obvious design | Being wrong is expensive and the failure is hard to spot. |
| Premium reasoning | Production incidents | The one class where cost does not enter the decision. |
Decision points
- Does this involve sensitive data? If yes, no prompt — before any tier is considered.
- Can this be answered deterministically? The most valuable question in the policy.
- What class of task is this?
- Are we within rate and budget? Incidents are exempt from downgrade.
- Is the tier available? Fall back on capability, never on approval rules.
Responsibilities
AI / LLM
- Nothing. Routing is decided before a model is involved, and no model chooses its own tier.
Deterministic
- Sensitivity classification.
- The deterministic-fact test.
- Task classification and tier selection.
- Rate and budget checks.
- Spend attribution.
- Fallback selection.
Human approval points
- Setting the routing policy and revising it from tracked spend.
- Overriding a route for a specific task, which is recorded.
- Adjusting budget ceilings per task class.
Failure handling
| Failure | Response |
|---|---|
| Task mis-routed to too weak a tier | Output quality is visibly poor; retry at a higher tier and record the correction. |
| Task mis-routed to too strong a tier | Wasted spend, caught by attribution rather than at the time. |
| Budget ceiling reached | Defer or downgrade — except incidents, which are never downgraded. |
| Primary tier unavailable | Defined fallback per tier. The approval boundary never changes. |
| Sensitive data in a task | Blocked before context assembly. Not a routing decision at all. |
Monitoring
Spend per task class, retry rate by tier, mis-routing corrections, budget exhaustion, and fallback activation. Rising corrections at a tier means the classification rules need work, not that the tier is wrong.
Business value
Cost fell and quality rose simultaneously, because the two problems had the same cause — treating every task as though it were the same difficulty.
Related projects
- Using AI to Improve Product DevelopmentStatus: Live internal tool
The development process behind the other case studies — written requirements, scoped tasks, AI coding tools, automated pre-release checks and human approval on every production change.
- AI Agent Governance and Knowledge PlatformStatus: Partially operational
The controls that let AI agents work against a live system — spending limits, an explicit list of permitted actions, a staged rollout, and search that grounds them in real documentation.
Related workflows
- PRD to Production
From a written requirement to a verified production change — context packs, task cards, deterministic commit gates, AI review before push, human approval to deploy.
- Incident Diagnosis and Approval
Read-only AI diagnosis during a production incident, with a human approving every action — because the fastest wrong fix is worse than a slower right one.