Cursor's Coding Model Wasted 50%+ of Every GPU-Hour. The Fix Is a Free Kernel, Not More Chips.

The industry's alibi, and the receipts that contradict it

Every AI lab tells you a version of the same story when training costs come up: they can't get enough GPUs, Nvidia can't ship fast enough, compute is the bottleneck. 2026 AI infrastructure spend is tracking toward $401 billion industry-wide, and most of that number gets waved at as proof the constraint is physical: not enough silicon.

Cursor Research's “Mixture-of-Kittens: our open-source MoE megakernel for NVL72s”, published August 4, 2026 by Stuart Sul, Nash Brown, Henry Wildermuth, William Lin, and Federico Cassano, tells a different story with their own training logs as evidence. They train Composer, Cursor's coding model, on Nvidia's newest GB300 NVL72 racks: 72 GPUs in a single NVLink domain, wired at 1.8 terabytes per second. Even with that much bandwidth on tap, they write plainly that “communication had become the limiting factor” in the mixture-of-experts layer, and that it was “consuming more than half of end-to-end training time” depending on workload configuration. Not computing. Not fetching data. Talking, chip to chip, while the actual math waited.

Where the other half of the GPU-hour actually goes

A mixture-of-experts model routes each token to a small subset of specialized sub-networks (“experts”) instead of running every parameter on every token. That's the efficiency trick that makes trillion-parameter models trainable at all. It also means every layer needs a dispatch step, shipping each token to whichever GPU holds its assigned expert, then a combine step, gathering the results back. On a single GPU, both steps are free. Spread the experts across 72 GPUs in a rack, and both steps become network operations, and network operations have historically been treated as someone else's problem to optimize, separately, after the math is done.

The root cause: 2 teams, 1 gap nobody owned

The paper trail Cursor publishes is almost embarrassing in how simple the root cause turns out to be. Every optimization effort before this one treated computation and communication as 2 separate problems: solved separately, by separate teams, run sequentially at runtime. Compute a layer, then wait for the network to finish moving tokens, then compute the next layer. Nobody on either side owned the gap between the two phases, and that gap is exactly where the missing GPU-hours went. It is the kind of bottleneck that survives for years specifically because it sits at an org-chart seam instead of inside one team's dashboard.

New hardware, new bottleneck: the idle-CPU problem

The GB300 NVL72 generation made the underlying issue more visible, not less. Its integrated Grace CPUs run slow relative to its Blackwell GPUs, and Cursor's engineers found that “GPU streams easily caught up to CPU-side work,” leaving expensive accelerators idle while a comparatively cheap CPU finished scheduling work for them. A rack built to remove bottlenecks introduced a new one, just one layer up the stack.

The fix: fuse compute and communication into 1 kernel

Cursor's answer is a deterministic megakernel they call Mixture-of-Kittens (MoK). Instead of launching a compute kernel, waiting on it, then launching a separate communication kernel, MoK fuses both into a single kernel launch. Some of the chip's streaming multiprocessors (SMs) are assigned to computation, others to communication, and both run concurrently, signaling completion to each other through local counters instead of blocking on kernel-launch boundaries. The design also adds tunable minibatch sizing (a heuristic of at least 2 full SM waves per grouped GEMM), fixed-size ring token buffers to eliminate CPU-GPU synchronization entirely, and native MXFP8 quantization fused directly into the dispatch and GEMM steps. Every run is bitwise-deterministic, a detail that matters for anyone debugging a multi-week training run.

The counterintuitive call: pull beats push

One design decision breaks with standard practice outright. The conventional approach pushes data outward to the next GPU that needs it. Cursor's engineers tested pulling data instead, having each GPU request what it needs rather than wait to receive it, and pull-based dispatch delivered up to 29% higher NVLink bandwidth utilization under expert imbalance. The industry default was the slower choice the entire time, and almost nobody had benchmarked the two approaches head to head before this.

Benchmarked 4 ways, 4 times over

Cursor tested MoK as a single MoE layer against the fastest available public baselines, NCCL+PyTorch, DeepEP variants, and HybridEP+Megatron, across 4 different model shapes: Kimi K2.7 Code, GLM-5.2, Qwen3.5-397B-A17B, and DeepSeek-V4-Pro. The result lands the same direction in every configuration tested.

These are single-layer microbenchmarks, not marketing round numbers. Cursor also reports the schedule kernel itself takes under 3% of total MoE layer runtime, meaning almost none of the recovered time is eaten by the fix's own bookkeeping.

What 41% actually costs, and what it's worth

Microbenchmarks are one thing. The number that changes a real training budget is the production result: running MoK on 512 GPUs across multiple NVL72 racks, Cursor measured throughput rising from 760.9 tokens per second per GPU to 1,070.2, a 41% gain, compared to their previous DeepEP-based production setup. Same hardware. Same power bill. Same GPU lease. The only variable that changed was the kernel.

To put a price on that, CoreWeave lists GB200 NVL72 on-demand access at $10.50 per GPU, per hour, the closest public reference rate available since CoreWeave doesn't list GB300 pricing publicly. Recovering 41% of useful throughput on a training run spanning thousands of GPUs over multiple weeks isn't a rounding error against a bill at that rate, it's a meaningful fraction of the entire run's compute cost, recovered without buying, renting, or waiting on a single additional chip.

They gave it away: what's actually in the GitHub repo

Cursor released MoK under an Apache 2.0 license at github.com/cursor/mixture-of-kittens, at 494 stars and 50 forks 6 days after publishing, with the benchmark code included rather than just the headline numbers. It targets Nvidia Blackwell SM100 or SM103 GPUs (GB200 or GB300 NVL72), Python 3.12+, PyTorch 2.10+, and CUDA 13.0+:

# Standard install
pip install . --no-build-isolation

# SM100-specific build (GB200 NVL72)
MOK_ARCH=SM100 pip install . --no-build-isolation

The authors are candid about how they built it this fast: AI coding agents handled the simpler kernel variants and let a small team “get through the harder ones… much faster and with a very small team.” Their stated hope is that releasing the full source “lowers the barrier to AI research, allowing more researchers and labs to train models efficiently,” not just Cursor.

How to check whether your own training run has the same leak

You don't need Cursor's scale to have Cursor's problem. Any MoE training job spread across multiple GPUs is paying some version of the same communication tax, the only question is how much. Before reaching for MoK specifically, the fastest way to find out where your own run stands is profiling the compute-versus-communication split directly with PyTorch's built-in profiler:

import torch
from torch.profiler import profile, ProfilerActivity

with profile(
    activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA],
    record_shapes=True,
) as prof:
    train_step()

# Look for time spent in nccl / all_to_all / dispatch ops
# versus matmul / gemm ops in the trace
print(prof.key_averages().table(sort_by="cuda_time_total", row_limit=20))

If NCCL collectives (all_to_all, all_reduce) and dispatch or combine ops are eating a large share of total CUDA time relative to your GEMM ops, you're looking at the exact shape of bottleneck Cursor documented, and it's worth checking whether MoK's architecture, or at minimum its pull-based dispatch finding, applies to your own setup before assuming the fix is simply more GPUs.

The takeaway

The AI industry's default explanation for high training costs is a GPU shortage. Cursor's own training logs say the more expensive shortage is attention: nobody owned the gap between the compute team and the networking team, so it sat there for years, quietly consuming more than half of every GPU-hour on some workloads. The fix that recovered 41% of it wasn't a bigger cluster. It was one team asking a question nobody upstream had bothered to ask, then giving the answer away for free.

If you're paying for GPU time and have never checked what fraction of it is actually computing anything, MegaBrainroutes your inference workloads through a single gateway with transparent, at-cost pricing across 500+ models, so at least the layer you don't control isn't hiding the same kind of tax. And if you want an agent that keeps watching infrastructure costs like this while you sleep, mhermes runs 24/7 on its own isolated VM and can be scheduled to track exactly this kind of story, new benchmark releases, GPU pricing changes, open-source infra tools, and flag the ones worth your attention.