Merit Order
GitHub

Compare

How it compares, with sources.

Nine other gateways and routers, checked against their public docs on 15 November 2026. Where others are ahead is below the table. This page is the feature sheet's comparison, pulled from the main repository at build time.

Comparison table

Checked on 15 November 2026 against each product's current public docs. "Partial" means the product does part of it, or does it a different way; the numbered notes below the table say how, with sources. A ✓ for Merit Order means built and tested in CI, which for most rows means against mocks (Honest limits).

CapabilityMerit OrderLiteLLM proxyOpenRouter Auto RouterCloudflare AI GatewayPortkeyNot DiamondRouteLLMKong AI GatewayRequestyHelicone
Quality-floor routing from the work✓partial (1)✓ (2)✓, beta (3)✗ (4)✓ (5)partial (6)partial (7)partial (8)✗ (9)
Cache-aware routing✓partial (1)✓ (2)✓ (3)✗✗✗partial (7)partial (8)✗
Cost per success✓, opt-in✗✗partial (3)✗✗✗✗✗✗
Learned router from your outcomes✓, opt-in (10)partial (1)✗ (2)✗✗✓ (5)partial (6)✗✗✗
Self-hosted GPU autoscaling at break-even✓ (11)✗✗✗✗✗✗✗✗✗
Bill reconciliation✓ (12)partial (13)n/a: it is the bill✗✗n/a✗✗✗✗
Savings attribution by cause✓partial (13)✗✗✗✗✗✗✗✗
Self-hostable✓✓✗✗✓partial (5)✓✓✗✓
Open source✓ AGPL-3.0✓ MIT core (14)✗✗✓ MIT (4)✗✓partial (7)✗✓ (9)

Martian isn't in the table: its documentation now lists a gateway and other products, and its router pages return 404 (docs.withmartian.com). We couldn't confirm with Martian whether the router is still offered.

What this shows. Of the tools we checked on 15 November 2026, none combines most of these rows. But several rows aren't unique to Merit Order:

  • Cache-aware, task-aware routing already exists. OpenRouter's Auto Router and Cloudflare's Auto Router (beta) both read the request to judge the task, and both weigh the cost of losing a prompt cache before switching models. OpenRouter also chooses reasoning effort per request. Both are hosted services you can't run yourself.
  • Not Diamond trains routers on your own data as a product, with more history than Merit Order's learned router. LiteLLM's adaptive mode learns from feedback within a tier.
  • Cloud platforms have learned routers too. Azure's Foundry model router picks among Azure models with a trained model, in balanced, cost and quality modes; its session affinity doesn't read cache signals (how it works). AWS Bedrock's Intelligent Prompt Routing switches between two models of one family when the predicted quality gain passes a threshold, and its docs say it can't adjust to your application's own performance data (docs). The open-source vLLM Semantic Router combines classifier signals with rules and a semantic cache.

What we found nowhere else: starting and stopping your own GPUs from what the traffic would cost at API prices, savings split by cause so they add up, and checking costs against several providers' bills (LiteLLM does OpenAI's). Cost per success as a routing objective appears only in Merit Order; Cloudflare reports it as a benchmark result.

Where others are ahead.

  • A hosted service. OpenRouter and Cloudflare run at very large scale, with nothing to deploy. Merit Order is something you operate: Postgres, Redis, a worker, and optionally a GPU controller.
  • Maturity and production use. LiteLLM, Kong, Portkey and OpenRouter are used in production by many organizations. Merit Order is 0.0.1, and most of it has been tested only against mocks.
  • Provider breadth. LiteLLM covers over 100 providers and OpenRouter hundreds of models. Merit Order has 14 provider APIs plus any OpenAI-compatible server.
  • Enterprise features. Kong and Portkey (now part of Palo Alto Networks) have guardrails, API management and security features Merit Order doesn't try to have.

Notes

  1. LiteLLM's Auto Routing (beta since v1.94.0) puts each request in one of four tiers with a heuristic scorer whose dimensions include prompt length, or with an LLM classifier, keyword rules or a plugin. Tiers map to models you choose; there are no quality scores per model. Session affinity (off by default) and a prompt-caching check keep a session or a cached prefix on one deployment, but a move's rebuild cost isn't priced. With adaptive: true it learns within a tier's pool by Thompson sampling, crediting feedback to the model that served (auto routing).
  2. A decision model rates each request on eleven aspects, and the policy "weighs that rating against live prices, cache status, and benchmark performance" within a cost_tier. "Switching models costs extra while the new model rebuilds the prompt cache, so the router is restrained about it", and it also picks a reasoning effort per request. It learns from OpenRouter's aggregate usage, not from your outcomes (Auto Router).
  3. A classifier estimates the task's type and difficulty; a scoring step ranks models by expected quality against cost, and it "switches models only when the expected benefit outweighs the cost of losing the cache". Its launch benchmark reports cost per success, but that's a published result, not a routing objective you can set (Auto Router docs, launch post).
  4. Conditional routing matches metadata, request parameters and the URL path with operators such as $eq and $regex, not the prompt's content. The gateway is MIT; semantic caching is in the hosted and enterprise versions. Palo Alto Networks acquired Portkey in 2026 (conditional routing, GitHub).
  5. Pre-trained routers for chat and code, and custom routers trained on your prompts, responses and scores. A router, not a gateway. VPC deployment on enterprise tiers. Shadow mode isn't documented (docs, router training, pricing).
  6. A research framework: learned routers between one strong and one weak model, trained on Chatbot Arena preference data, which you can retrain yourself (GitHub).
  7. The semantic algorithm matches prompt embeddings to model descriptions you write; consistent hashing gives header-based stickiness. AI Proxy Advanced is part of the enterprise offering, though Kong Gateway itself is open source (AI Proxy Advanced).
  8. An LLM you choose classifies the first user message against route labels you write, with a confidence threshold. A conversation's route is pinned for 60 minutes so prompt caches keep working; a switch's cost isn't priced (auto routing).
  9. Latency-based, weighted and cheapest-option load balancing; no routing on the work. Mintlify acquired Helicone in March 2026, and the gateway is in maintenance mode. The repository is open source; its README and repository metadata name different licenses (GitHub, announcement).
  10. Shown on synthetic traffic only; how it does on real traffic is what its shadow report is for.
  11. Tested on a local kind cluster with a stand-in vLLM, not yet on real GPUs.
  12. Seven bill sources. Only OpenRouter's costs have been compared with a real bill; the others run against mocks.
  13. "Spend capture rate" compares tracked spend with OpenAI's Organization Costs API; other providers are compared by hand (spend capture rate). Its auto-routing benchmark reports savings against a baseline, cache reads and writes included for supported Anthropic requests, as one figure rather than split by cause.
  14. MIT core with a commercial enterprise license for some features (GitHub).