Merit Order
GitHub

Savings

Where the money comes from.

Each mechanism, what it saves with a worked example, when it doesn't help, and how the ledger measures it. Then the routing simulator's results and a calculator that scales them to your team. Prices of 12 November 2026; yours will drift as the price list refreshes.

Mechanisms

routing simulator

Routing by expected total cost

What it does. Gives each task a level from its wording (tools only say it is coding), reads each model's chance of finishing work at that level from published benchmarks, and sends the task to the model and route with the lowest expected total cost: the try, plus the chance it fails times a person's time and the next attempt.

Where the money comes from. Most tasks don't need the strongest model, and a cheap model that fails costs more than it saves. On the simulator's 2,000 standard tasks, the default spent $0.40 on models per finished task, against $0.73 with Claude Sonnet 5.5 for everything and $1.43 with Claude Opus 5.5, and finished 97.7% of tasks against 97.4% and 97.7%.

When it doesn't help. When most of your work is at the top level, where only the dearest models have a good chance. On coding-only work, cost per success and the default come out about even once failed attempts are counted ($3.00 and $2.93 per finished task). The level rules are word lists, so a listed word can raise a task that isn't hard ("security" in "update the security page's footer"). And the chances come from benchmarks, not yet from your own outcomes.

How it's measured. Offline, by the routing simulator (below). Live, by the "routing" cause in the ledger: the baseline model's cost minus the served model's, at list prices, before the other causes, and by the outcome report.

list-price arithmetic

Prompt caching

What it does. Adds Claude's cache markers when the client set none, sets OpenAI's prompt_cache_key, and never changes the early part of the prompt. On the four Claude backends it asks for a 1-hour cache from a session's second turn.

Where the money comes from. Cached tokens cost a tenth of the input price or less. On a 100,000-token conversation, the hour costs $0.0030 more per turn; after a 20-minute pause the next turn reads the prefix for $0.010 instead of writing it again for $0.25.

When it doesn't help. When the harness already caches well. Claude Code sets its own markers, and caching a client set up itself isn't counted as a Merit Order saving. An agent that never pauses pays the hour's premium for nothing (about 20% more per turn on Sonnet and Opus 5.5).

How it's measured. The "prompt caching" cause, only for caching the gateway added. Checked live: gateway-added Claude caching, turns 2 and later at a tenth of the price. Not yet checked against an Anthropic invoice.

list-price arithmetic

Sticky sessions

What it does. Keeps a session on the route that holds its cache unless moving saves at least 10% plus $0.01 over the rest of the session. Within a task it keeps its model outright.

Where the money comes from. Rebuilding an 80,000-token Claude Opus 5.5 session on a cold backend costs $0.608, against $0.044 for a turn where it is. Stickiness stops a session bouncing between Bedrock and Anthropic.

When it doesn't help. It saves nothing on one-off requests. Sessions that switched backends, and the rebuild cost they paid, aren't reported yet.

How it's measured. Indirectly: it shows up as cache reads in "prompt caching" and as lower costs per turn.

illustration · opt-in

Tool-output compression

What it does. Deterministic rules strip terminal codes and trailing spaces, collapse repeated lines, and keep the first 60% and last 40% of tool output over 24,000 characters. An optional model (LLMLingua-2) drops the least useful words from prose, protecting paths, numbers, identifiers and error lines.

Where the money comes from. A 100,000-character test log becomes about 24,000 characters: roughly 25,000 fewer tokens. On Claude Sonnet 5.5 that is $0.10 less on the turn it arrives and $0.0025 less on every later turn.

When it doesn't help. On short tool output (under 4,000 characters is left alone), and the model-based method removes nearly nothing from code, logs and JSON (5 to 13% of a web page or docs). Whether agents still succeed on compressed results hasn't been measured.

How it's measured. Two causes: "compression (rules)" and "compression (model)".

illustration

Batch prices and GPU headroom

What it does. Accepts OpenAI's Batch API and Anthropic's Message Batches, routes each request like a live one among providers with a batch API and running GPUs with room, then joins the results.

Where the money comes from. Batch APIs cost about half the live price (Moonshot 40% off, xAI 20%). 10,000 summaries of 5,000 tokens in and 500 out on Claude Sonnet 5.5: $150 live, $75 at the batch price.

When it doesn't help. For interactive work, which can't wait up to a day. And you could get the batch discount from the provider directly; the ledger records it as its own cause for that reason.

How it's measured. The "batch discount" cause. Batch APIs are tested against mocks of ten providers, not yet the real ones.

shipped configuration

Your GPUs at break-even

What it does. Starts a replica when the traffic it could serve would cost more than its hourly price plus 20% at API prices for 10 minutes, and drains it when demand falls below break-even. While it runs, the router prices it at 2% of the API price up to 60% load.

Where the money comes from. Steady load is cheaper on a GPU you already pay for. The shipped Qwen3 Coder Next deployment (four H100s, $16 an hour) starts above $19.20 an hour of such work.

When it doesn't help. Today, mostly. On 11 November 2026 spot prices only level-1 work could pay for a GPU, because cheap API models set a high bar. Real GPUs on a real cloud haven't been tested yet.

How it's measured. The "self-hosting" cause; GPU time is compared with cloud bills by the deployment tag.

illustration · opt-in

Response cache

What it does. An exact-match cache of responses, per team or per request, keyed by team so answers never cross teams.

Where the money comes from. A pipeline sending the same $0.02 request 5,000 times a day, with a one-hour TTL, pays about 24 times a day ($0.48) instead of 5,000 ($100).

When it doesn't help. For agents: their turns are almost never repeated. It stores content, so it's off unless a team turns it on.

How it's measured. The "response cache" cause: a hit costs $0 and records the original cost as the saving.

shipped configuration

Budgets and runaway caps

What it does. Monthly budgets for the organization, teams and people, a per-session cap and a per-request ceiling, all checked before a request is sent.

Where the money comes from. Not a saving on any one request: it caps the bad day. A request for 128,000 output tokens on Claude Fable 5.2 with 500,000 tokens in has a worst case of $9.12, under the shipped $25 ceiling. A team's max_session_usd: 20 stops a looping agent at $20, its subagents included.

When it doesn't help. It doesn't make good traffic cheaper.

How it's measured. Refusals say which limit they hit; alerts go out at 80% and 100%.

Savings by cause, so they add up

Every ledger row has the actual cost, the cost on the team's baseline model (baseline_model, Claude Sonnet 5.5 in the shipped config) at API prices without the caching Merit Order added, and savings split into seven causes. The causes are computed in a fixed order, each against the result of the previous one, so there is no double counting. Non-streamed replies also carry x-meritorder-cost-usd, x-meritorder-baseline-usd and x-meritorder-savings-usd.

Checked against your bills

The worker reads each provider's own cost data every 6 hours: Anthropic's and OpenAI's cost APIs, AWS Cost Explorer or CUR 2.0, the Google Cloud billing export, and Azure Cost Management. It compares them with the ledger by provider, model and day, and alerts when a settled day differs by at least $1 and 5%. So far only OpenRouter's recorded costs have been compared with a real bill (they matched for every request); the other sources run against mocks.

Routing, simulated

What the routing simulator says.

Every number about routing on this site comes from scripts/routing_sim.py in the main repository, run on its current code (09750a2, 16 November 2026). It is a simulation: no model is called, and none of this is measured on live traffic yet.

How it works

The simulator draws tasks from a population of seven classes, from short chat to very hard coding, each with a difficulty, a number of turns and a context size, sent by Claude Code, a small agent such as Codex, or chat. Each strategy routes every task with the gateway's own router code. Whether an attempt succeeds is drawn from the model's published benchmark ability against the task's difficulty; a failed attempt is retried, up to 3 attempts, and is priced at $15 of someone's time (noticing, asking again, waiting, cleaning up). Tokens are priced on the main repo's price list of 12 November 2026, with cache writes, reads and rebuilds after pauses (10% of later turns).

"Always" strategies send every task to one model. The oracle knows in advance which cheapest model would succeed: no router can do that, so it is a lower bound.

Where it doesn't win

  • On coding-only work, cost per success spends less on models than the default ($0.46 against $0.47 per finished task) and the two are about even in total, counting failed attempts ($3.00 and $2.93).
  • On the standard mix the default spends more on models than cost per success ($0.40 against $0.40); it wins by finishing more tasks (97.7% against 96.8%), which only shows once failed attempts are counted.
  • On short chat it often picks a dearer model than it needs: fractions of a cent a task, but more than the threshold.
  • Escalating inside a task (opt-in) finished 97.8% of tasks against 97.7%, but at $3.08 against $3.08 per finished task in total, counting failed attempts.
  • The oracle needs $0.17 of model spend per finished task against the default's $0.40: routing that knew each task's difficulty in advance could still do much better.

2,000 tasks, the standard mix

72% coding, the rest chat and analysis: a gateway in front of coding agents and chat. Seed 1.

Simulated results by strategy, 2,000 tasks, the standard mix
StrategyFinishedModels, per finished taskTotal, counting failed attempts90% interval of the totalModels, vs Sonnet
Claude Opus 5.5 for everything97.7%$1.43$3.95$3.57 to $4.3595% more
Claude Sonnet 5.5 for everythingalso a fixed-model proxy set to Sonnet: the same tokens97.4%$0.73$3.40$3.08 to $3.76
The cheapest model for everythingDeepSeek V4.1 Flash94.6%$0.020$4.17$3.74 to $4.6497% less
auto, threshold (the default until decision 62)97.5%$1.01$3.63$3.20 to $4.0738% more
auto, cost per successopt-in96.8%$0.40$3.47$3.12 to $3.8646% less
auto, expected total cost (the default)97.7%$0.40$3.08$2.77 to $3.4345% less
auto, expected total cost, with one escalationopt-in (routing.cascade)97.8%$0.54$3.08$2.77 to $3.4526% less
Oracle: the cheapest model that would have succeededa lower bound: no router knows in advance98.7%$0.17$2.03$1.78 to $2.2876% less
By task class, the default against Sonnet for everything
ClassTasksDefault: finishedper finished taskSonnet: finishedper finished task
Coding, easy559100.0%$0.027100.0%$0.192
Coding, medium740100.0%$0.218100.0%$0.762
Coding, hard12880.5%$4.9075.0%$6.07
Coding, very hard1060.0%$17.5560.0%$25.06
Short chat159100.0%$0.0100.0%$0.0029
Chat278100.0%$0.0022100.0%$0.027
Analysis12686.5%$0.03387.3%$0.134

2,000 coding tasks

Claude Code 75%, a small agent such as Codex 15%, chat with code 10%: the ten developers on the home page. Seed 1.

Simulated results by strategy, 2,000 coding tasks
StrategyFinishedModels, per finished taskTotal, counting failed attempts90% interval of the totalModels, vs Sonnet
Claude Opus 5.5 for everything98.3%$1.79$4.16$3.79 to $4.5794% more
Claude Sonnet 5.5 for everythingalso a fixed-model proxy set to Sonnet: the same tokens97.9%$0.92$3.50$3.16 to $3.88
The cheapest model for everythingDeepSeek V4.1 Flash96.1%$0.024$3.67$3.29 to $4.1097% less
auto, threshold (the default until decision 62)98.5%$1.02$3.36$3.00 to $3.7912% more
auto, cost per successopt-in98.2%$0.46$3.01$2.70 to $3.3850% less
auto, expected total cost (the default)98.1%$0.47$2.93$2.62 to $3.3149% less
auto, expected total cost, with one escalationopt-in (routing.cascade)98.3%$0.59$2.98$2.67 to $3.3735% less
Oracle: the cheapest model that would have succeededa lower bound: no router knows in advance99.0%$0.16$2.05$1.84 to $2.3082% less
By task class, the default against Sonnet for everything
ClassTasksDefault: finishedper finished taskSonnet: finishedper finished task
Coding, easy767100.0%$0.026100.0%$0.202
Coding, medium1,054100.0%$0.237100.0%$0.764
Coding, hard16881.5%$3.9479.8%$5.20
Coding, very hard1136.4%$27.5227.3%$50.38

Counting the cost of failed attempts

A failed attempt costs someone time: noticing it, asking again, waiting, and cleaning up. Here that time has a price per failed attempt. The price is an assumption you can change. The routing doesn't change with it; only the totals do. Per finished task, on the standard mix.

Price per failed attempt
  • Model spend
  • Time lost to failed attempts
  • The default, highlighted
Claude Opus 5.5 for everything97.7% of tasks finished
Claude Opus 5.5 for everything: $1.43 model spend plus $2.53 time lost to failed attempts, $3.95 in total per finished task; 97.7% of tasks finished.
Claude Sonnet 5.5 for everything97.4% of tasks finished
Claude Sonnet 5.5 for everything: $0.73 model spend plus $2.66 time lost to failed attempts, $3.40 in total per finished task; 97.4% of tasks finished.
The cheapest model for everything94.6% of tasks finished
The cheapest model for everything: $0.02 model spend plus $4.15 time lost to failed attempts, $4.17 in total per finished task; 94.6% of tasks finished.
auto, threshold (the default until decision 62)97.5% of tasks finished
auto, threshold (the default until decision 62): $1.01 model spend plus $2.62 time lost to failed attempts, $3.63 in total per finished task; 97.5% of tasks finished.
auto, cost per success96.8% of tasks finished
auto, cost per success: $0.40 model spend plus $3.07 time lost to failed attempts, $3.47 in total per finished task; 96.8% of tasks finished.
auto, expected total cost (the default)97.7% of tasks finished
auto, expected total cost (the default): $0.40 model spend plus $2.67 time lost to failed attempts, $3.08 in total per finished task; 97.7% of tasks finished.
auto, expected total cost, with one escalation97.8% of tasks finished
auto, expected total cost, with one escalation: $0.54 model spend plus $2.54 time lost to failed attempts, $3.08 in total per finished task; 97.8% of tasks finished.
Oracle: the cheapest model that would have succeeded98.7% of tasks finished
Oracle: the cheapest model that would have succeeded: $0.17 model spend plus $1.85 time lost to failed attempts, $2.03 in total per finished task; 98.7% of tasks finished.

Every router and fixed model here loses at least $2.53 per finished task to failed attempts, and even the oracle loses $1.85. Most of it comes from vague first prompts that no model can fix, which is why the bars look alike. Opus loses the least time to failures ($2.53) but costs the most to run ($1.43, 1.9 times Sonnet's $0.73). The default has the lowest total short of the oracle: it spends $0.40 on the model, far less than Sonnet, and loses about as little time ($2.67 against $2.66). It cuts model spend without wasting people's time.

Per finished task at $15 per failed attempt, standard mix
StrategyModel spendTime lost to failed attemptsTotalTasks finished
Claude Opus 5.5 for everything$1.43$2.53$3.9597.7%
Claude Sonnet 5.5 for everything$0.73$2.66$3.4097.4%
The cheapest model for everything$0.02$4.15$4.1794.6%
auto, threshold (the default until decision 62)$1.01$2.62$3.6397.5%
auto, cost per success$0.40$3.07$3.4796.8%
auto, expected total cost (the default)$0.40$2.67$3.0897.7%
auto, expected total cost, with one escalation$0.54$2.54$3.0897.8%
Oracle: the cheapest model that would have succeeded$0.17$1.85$2.0398.7%

The main repository's Offline vs live and How we measure routing explain the simulator and how it will be checked against real outcomes. Commands: uv run python scripts/routing_sim.py run --tasks 2000 --mix standard and uv run python scripts/routing_sim.py run --tasks 2000 --mix coding-only.

Calculator

Scale the simulation to your team.

Takes the simulator's results above and scales them to your number of tasks and your price per failed attempt. It doesn't model your traffic: if your work differs from the simulated population, so will your savings. Nothing you enter leaves the page.

Team
Work and people
The simulated population
A failed attempt costs someone time: noticing it, asking again, waiting, cleaning up. The simulator's default is $15: about 10 minutes at $90 an hour.

The calculator needs JavaScript. The ten-developer month at $15 a failed attempt: $2,947 on Opus, $1,518 on Sonnet, $773 with the default, in model spend.

Monthly spend by strategy
Method
tasks a month   = developers × tasks a day × working days
model spend     = the run's model spend ÷ the run's tasks × tasks a month
time lost       = the run's failed attempts ÷ the run's tasks × tasks a month × price
finished tasks  = the run's share finished × tasks a month

The same scaling gives the home page's 1,680-task month. The price per failed attempt changes only the accounting; the router's own setting for it (human_cost_usd, $1 by default) is fixed in the run. Not counted: compression, batch, the response cache, GPUs, and the cost of running the gateway.

Simulated on 16 November 2026 from the main repository at 09750a2, prices of 12 November 2026.