AMD Matches Nvidia on Spec Sheets. On Real Agent Workloads, It Loses By Up To 20x.

The benchmark every GPU comparison was missing

Almost every public GPU inference benchmark tests the same shape of request: a fixed input length, a fixed output length, one turn, no history. That shape describes a chatbot answering a single question. It does not describe what an always-on coding agent does, which is hold a session open for tens to hundreds of turns, reuse most of its context from one turn to the next, and spawn sub-agents mid-task. SemiAnalysis built AgentX specifically to close that gap: it replays real agent traffic and scores hardware on throughput and cost at the interactivity level a live session actually needs, not on a synthetic prompt chosen for convenience.

The dataset behind it is real production traffic. SemiAnalysis intercepted and anonymized over 8,000 Claude Code and Codex sessions totaling 610 billion tokens, then open-sourced a 393-session subset under an Apache 2.0 license so anyone can reproduce the numbers. The firm reports this cost roughly $3 million to assemble and validate, with accuracy checks run against the original logs at every point in the replay, not just at the end.

That input-length spread matters more than it looks. A benchmark built around an 8K fixed prompt is testing a completely different serving regime than one where a quarter of sessions exceed 272,000 input tokens and some run past 675,000. Serving that workload well depends on how efficiently a stack caches and reuses that context across turns, which is exactly the thing a single-turn benchmark cannot measure at all.

Where the gap actually shows up

AgentX measured three named models across both vendors' current hardware, at different interactivity targets (the tokens/sec/user rate a real session needs to feel responsive). The multipliers below are cost-efficiency, meaning tokens served per dollar of hardware, not raw speed.

The DeepSeek V4 Pro numbers are the most revealing because SemiAnalysis measured them at two different interactivity targets on the same model. AMD's MI355X running SGLang was roughly matching Nvidia's B200 on performance-per-dollar before an August 21 software update; after it, Nvidia's B300 and B200 both pulled ahead again, and the gap grows as the target interactivity rises, from 57% at 108 tokens/sec/user to 247% at 141. A stack that looks competitive at a relaxed responsiveness target can fall apart at the tighter one a real coding agent actually wants.

# MegaBrain estimate, built from SemiAnalysis's published AgentX
# cost-efficiency multipliers. Not company-confirmed; ranges reflect
# the interactivity target each figure was measured at.

findings = {
    "GLM 5.3 @ 150 tok/s/user (SGLang)":        (5.00, 5.00),
    "DeepSeek V4 Pro @ 108-141 tok/s/user":     (1.57, 3.47),
    "Qwen3.5 397B, same generation (SGLang)":   (20.0, 20.0),
}

for model, (lo, hi) in findings.items():
    band = f"{lo:.2f}x" if lo == hi else f"{lo:.2f}x-{hi:.2f}x"
    print(f"{model}: AMD needs {band} more MI355X racks "
          f"to match Nvidia's agent throughput per dollar")

# GLM 5.3 @ 150 tok/s/user (SGLang): AMD needs 5.00x more MI355X racks to match Nvidia's agent throughput per dollar
# DeepSeek V4 Pro @ 108-141 tok/s/user: AMD needs 1.57x-3.47x more MI355X racks to match Nvidia's agent throughput per dollar
# Qwen3.5 397B, same generation (SGLang): AMD needs 20.00x more MI355X racks to match Nvidia's agent throughput per dollar

The 20x gap, and the 12x gap that is not about AMD at all

It is easy to read every number in this report as “AMD lost,” and mostly that is right. But the Qwen3.5 397B results actually contain two separate findings that get conflated in secondary coverage. The 20x figure is AMD's MI355X against Nvidia's GB300 NVL72 on SGLang, same generation, same point in time: a straight vendor-to-vendor loss. A different comparison in the same report, Nvidia's own B300 running FP4 against Nvidia's own prior-generation H100 on the identical task, finds a 12x performance-per-dollar gain. That second number has nothing to do with AMD. It is the product of four years of Nvidia hardware generations plus a move to a lower-precision number format, and it means part of what looks like “AMD's problem” in casual coverage is really “how fast Nvidia is compounding gains against its own older chips,” a bar AMD has to clear twice: once against Nvidia's current generation, and once against the rate Nvidia is improving.

Cache hit rate, not FLOPS, is the real battlefield

The mechanism SemiAnalysis points to is not raw compute. Well-tuned AgentX configurations hit 95%+ KV-cache hit rates, meaning the serving stack is reusing more than 19 of every 20 tokens of context from previous turns instead of recomputing them. At that hit rate, the bottleneck stops being how many FLOPS a chip has and becomes how well its software schedules memory and cache eviction across hundreds of concurrent, long-running sessions. SemiAnalysis is explicit that isolated hardware or kernel optimizations get “swamped, on an agentic trace, by the cache and scheduling variance,” and singles out AMD's ATOM inference engine, its purpose-built answer to exactly this problem, as still immature: “still has a long way to go to support agentic workloads” compared to the vLLM and SGLang stacks most of the Nvidia numbers were run on.

That is a software gap, not a silicon one, and software gaps close on a different timeline than hardware ones. It also means a benchmark run again in six months with a more mature ATOM release could show a meaningfully different picture. SemiAnalysis reports more than 70 upstream pull requests already filed against vLLM, SGLang, TensorRT-LLM, ATOM, AITER, Dynamo, LMCache, and Mooncake using AgentX as the target to optimize against, which is itself a sign the industry is treating this specific gap as fixable, not permanent.

What this doesn't prove

SemiAnalysis is upfront about the benchmark's limits, and they matter. The headline metric it uses to rank configurations, “E2E Normalized Interactivity,” is described by the authors themselves as “experimental and not perfect”: it “heavily penalizes high time-to-first-token and doesn't capture all the nuances of certain optimizations.” The public dataset is 393 sessions, all sourced from Claude Code and Codex specifically, so it says a lot about coding-agent traffic and nothing directly about, say, customer-support or data-pipeline agents with different context and turn-gap shapes. And every multiplier here is a snapshot from configurations benchmarked in a specific week; both AMD and Nvidia ship new software stacks constantly, and the DeepSeek V4 Pro numbers already moved once inside the same report after an August 21 update.

None of that erases the finding. It means the finding is narrower than “Nvidia beats AMD,” and closer to: on real, long-context, high-prefix-reuse coding-agent traffic, measured this week, on these specific models, Nvidia's software stack converts its hardware into more tokens per dollar than AMD's does, by a margin wide enough that AMD giving its chips away for free would not close it on GLM 5.3.

If your agents run 24/7, this is the number that matters

Most teams choosing infrastructure for an AI agent still shop the way you'd shop for a chatbot: they look at a chip's spec sheet or a single-turn benchmark, and call it done. AgentX is a reminder that the number that actually predicts your bill is the one measured on a session that looks like yours, tens of turns long, mostly reused context, running whenever the agent has work, not whenever a person happens to be looking at a screen. That is precisely the operating pattern an always-on agent produces, and it is the pattern this report shows most public benchmarks still get wrong.

MegaBrainroutes every one of those calls across 500+ models at zero markup, with the real per-model, per-call cost broken out instead of a single blended number at the end of the month, so a hardware-driven price or throughput shift on any model you route through shows up as a number you can actually see. And because the whole point of a benchmark like this is that agent traffic doesn't stop when a person logs off, mhermes, MegaBrain's always-on agent runtime, is built to keep running against that same kind of long, high-reuse session, on its own isolated cloud VM, around the clock instead of only when someone remembers to open a chat window.

Sign up at getmegabrain.com to see what your own agent traffic actually costs, model by model, instead of guessing from a spec sheet.