OpenRouter's Router Setting Called “Max” Scored 2.4% on Real Code. The Free Default Scored 30.4%.

A router setting called “max” was the cheap option

OpenRouter runs an Auto Router: send a prompt to openrouter/auto, and it picks the model for you. Until August 10, 2026, that router picked based on a hand-tuned, benchmark-weighted heuristic. Users could set a cost_quality_tradeoff parameter from 0 to 1, where 0 meant “don't optimize for cost, get me the best model available.” That's the setting this piece calls “old max.”

OpenRouter's own comparison, published alongside the new router's launch, benchmarked both settings on SWE-Atlas QnA, a real-software-task coding benchmark. The old router's default setting landed 30.4% accuracy, spending $463.73 across the full benchmark run. Turning the dial to “max,” the setting built to prioritize quality over cost, dropped accuracy to 2.4%, and it did that for less money: $205.52.

Read that twice. The setting whose entire purpose was to spend more and get more was both cheaper and roughly 12 times worse than the setting sitting right next to it. That isn't a rounding artifact on a small sample; it's a full benchmark suite, reported with error bars (2.4% ±0.0 versus 30.4% ±2.3) tight enough to rule out noise.

Checking if it's a fluke: the same break on banking agents

A single benchmark inversion could be an artifact of one quirky test set. OpenRouter also ran τ³-bench Banking, an agentic benchmark that simulates a customer-support agent working through multi-step banking tasks. Same pattern, different domain.

Old max scored 7.2%, less than a third of the default's 21.0%, and it cost $99.36 against the default's $320.04, a 69% discount for a benchmark result nobody would have wanted. Two different agentic domains, coding and banking, both show the identical shape: turning the “spend more, get quality” dial to maximum produced a cheaper, worse model pick. Whatever caused this wasn't specific to one task type. It was systemic to how the old router handled its priciest setting on anything requiring multi-step action.

The bug had a shape: it only hit tasks that require acting, not answering

The inversion doesn't show up everywhere. OpenRouter also benchmarked MMLU Pro (general knowledge) and WideSearch (live web search) under the same two settings, and both behaved the way a “max” setting should: pricier, and more accurate.

Knowledge questions and search retrieval are, structurally, one-shot: the model reads a question and produces an answer. Coding and banking-agent tasks are multi-step: plan, call a tool, check the result, adjust, repeat. The break was isolated to exactly that category. It's a reasonable guess, not a confirmed root cause since OpenRouter hasn't published the underlying model-selection logs, but the pattern lines up with a heuristic that was tuned and validated against static, largely single-turn evaluation sets, then quietly stopped generalizing once real usage skewed toward longer agentic workflows.

The fix, and the arithmetic that actually decides which one is cheaper

On August 10, 2026, OpenRouter shipped a new Auto Router built on a different principle entirely: instead of a hand-tuned heuristic, it ranks models by what the OpenRouter community, in aggregate, actually spends on for a given task type, over a trailing 7-day window, refreshed continuously across roughly 30 fine-grained task classes and 55 trillion tokens of weekly traffic. OpenRouter calls it “wisdom of the market.”

Run the same coding benchmark through the new router's max tier: 60.7% accuracy, versus old max's 2.4%. That's a 25x jump. It also costs more in raw dollars: $1,325.08 versus $205.52, a 6.5x higher sticker price. Taken alone, that looks like the new router just bought its way to a better score. The number that actually matters for a team paying the bill isn't the sticker price, it's the cost per task that actually got done.

# router_cost_per_success.py — cost per completed task, not cost per run
BENCHMARKS = {
    "MMLU Pro (Knowledge)": {
        "new_default": (0.852, 140.93), "old_default": (0.866, 393.34),
        "new_max":     (0.914, 255.71), "old_max":     (0.888, 449.61),
    },
    "Banking (Agents)": {
        "new_default": (0.206, 155.89), "old_default": (0.210, 320.04),
        "new_max":     (0.316, 168.41), "old_max":     (0.072,  99.36),
    },
    "WideSearch (Search)": {
        "new_default": (0.616,  30.75), "old_default": (0.531,  31.83),
        "new_max":     (0.619,  36.89), "old_max":     (0.548,  31.60),
    },
    "DSQA (Research)": {
        "new_default": (0.629, 276.00), "old_default": (0.432, 147.11),
        "new_max":     (0.630, 248.83), "old_max":     (0.423, 144.18),
    },
    "SWE-Atlas QnA (Coding)": {
        "new_default": (0.304, 297.23), "old_default": (0.304, 463.73),
        "new_max":     (0.607,1325.08), "old_max":     (0.024, 205.52),
    },
}

for name, tiers in BENCHMARKS.items():
    print(name)
    for tier, (acc, cost) in tiers.items():
        per_success = cost / acc
        print(f"  {tier:12s}  acc={acc*100:5.1f}%  cost=${cost:9.2f}  cost/success=${per_success:9.2f}")

$ python router_cost_per_success.py
SWE-Atlas QnA (Coding)
  new_default   acc= 30.4%  cost=$   297.23  cost/success=$   977.73
  old_default   acc= 30.4%  cost=$   463.73  cost/success=$  1525.43
  new_max       acc= 60.7%  cost=$  1325.08  cost/success=$  2183.00
  old_max       acc=  2.4%  cost=$   205.52  cost/success=$  8563.33

Banking (Agents)
  new_default   acc= 20.6%  cost=$   155.89  cost/success=$   756.75
  old_default   acc= 21.0%  cost=$   320.04  cost/success=$  1524.00
  new_max       acc= 31.6%  cost=$   168.41  cost/success=$   532.94
  old_max       acc=  7.2%  cost=$    99.36  cost/success=$  1380.00

At the max tier on coding, old max spent $205.52 to land 2.4% of tasks, about $8,563 for every task it actually finished. New max spent $1,325.08 to land 60.7%, about $2,183 per finished task. The version with the bigger invoice is nearly 4 times cheaper per unit of work that actually got done. The same holds on banking: new max's $532.94 per success beats old max's $1,380.00 by roughly 2.6x, even though new max's raw dollar spend is higher ($168.41 versus $99.36). Sticker price and cost-per-completed-task point in opposite directions on both benchmarks where the old router was broken.

The new router isn't a strict upgrade either

It would be a cleaner story if the market-driven router won everywhere. It doesn't. At default settings, MMLU Pro accuracy drops from the old router's 86.6% to the new router's 85.2%, a 1.4-point decline. Across all 5 categories OpenRouter tested, that is the single case where routing by what people actually spend on loses to routing by a curated, hand-tuned benchmark score.

Even in the one regression, the cost math still tilts toward the new router: at 85.2% accuracy for $140.93 versus the old router's 86.6% for $393.34, the new default costs about $165 per correct answer against the old default's $454, roughly 2.7 times cheaper per completed task despite the lower raw score. The regression is real. It's also small next to a cost gap that isn't.

Where the new router spent more, on purpose

The strongest single win in the whole comparison isn't free, and it's worth being precise about that instead of only counting the wins that were also cheaper. On DSQA, a research-task benchmark, the new router's default tier jumps from the old router's 43.2% to 62.9%, up 19.7 points. It does that by spending 88% more: $276.00 versus $147.11. Run the same cost-per-success math and the new router actually costs more per completed task here too, about $439 versus $341, a genuine tradeoff rather than a free lunch.

How to audit a router setting before you trust its name

The mechanism behind the new router is public: it classifies a prompt into roughly 30 task types, ranks models within that type by trailing 7-day spend share across OpenRouter's community, and applies your chosen cost tier on top of that ranking, with “sticky” behavior that avoids switching models mid-conversation unless the current pick stops being a leading choice for the task. None of that mechanism would have caught the old bug faster. What would have is exactly what OpenRouter did here by accident of publishing a before/after comparison: running the same benchmark suite through every cost tier your product exposes, before shipping the settings page, and again periodically after.

# settings_audit.py — sanity-check that "higher tier" actually means "higher score"
def audit_tier_ordering(results):
    """results: {tier_name: (accuracy, cost)}, ordered cheapest to priciest by design"""
    tiers = list(results.items())
    for (name_a, (acc_a, cost_a)), (name_b, (acc_b, cost_b)) in zip(tiers, tiers[1:]):
        if acc_b < acc_a:
            print(f"INVERTED: {name_b} ({acc_b:.1%}) scores below {name_a} ({acc_a:.1%})")
        if cost_b < cost_a:
            print(f"INVERTED: {name_b} (${cost_b:.2f}) costs less than {name_a} (${cost_a:.2f})")

# Old router, coding benchmark, in ascending intended-cost order:
audit_tier_ordering({
    "default": (0.304, 463.73),
    "max":     (0.024, 205.52),
})

$ python settings_audit.py
INVERTED: max (2.4%) scores below default (30.4%)
INVERTED: max ($205.52) costs less than default ($463.73)

Two lines of assertion logic against a benchmark run OpenRouter already had internally would have surfaced this before it shipped, or at least flagged it the first week real traffic diverged from whatever the router was tuned against. The underlying lesson generalizes past this one router: any product surface with a “more expensive = better” setting, from model routers to cloud instance tiers to CDN cache classes, is a claim, not a guarantee, and the only way to know if it's still true is to measure it against the thing it's supposed to buy.

Why this is worth watching if you route your own agent traffic

The specific bug here is OpenRouter's, and by the numbers above, largely fixed as of August 10, 2026. The pattern it exposes isn't specific to OpenRouter. Any routing layer, whether it's a vendor's built-in auto-router or a custom classifier a team builds in-house, encodes assumptions about which model wins at which price point, and those assumptions decay as new models ship and usage patterns shift. A router that isn't re-validated against live benchmarks on a schedule is a router running on last quarter's truth, and the failure mode isn't a crash, it's a quiet, plausible-looking wrong answer that costs the same or more than the right one.

That's exactly the layer MegaBrainsits at: routing to 500+ models through one API at transparent, zero-markup pricing, so the cost side of “is this setting actually buying me anything” is never hidden behind a vendor markup you can't see. For agents that need to keep running and keep getting re-routed to whatever model is actually winning on a given task, not whichever one a stale heuristic still favors, mhermes, MegaBrain's always-on agent runtime, runs unattended on your schedule instead of a fixed, aging assumption about which model is “the good one.”

Sign up at getmegabrain.com to route at cost across every model that matters, or spin up a mhermes agent to keep watching your own router's numbers after this one.