Qiushi Engine Beat Claude Code to #1 on ResearchClawBench. It Still Can't Reliably Discover Anything New.

3,242 LLM calls. 145.9 million tokens. One physical finding validated so far. That's what it took for Zhejiang University's Qiushi Engine to reach the #1 spot on ResearchClawBench, the benchmark for autonomous end-to-end research, with Claude Code ranking third as of July 21. The South China Morning Post's own reporting on the result: the system that just topped the leaderboard still can't reliably make new discoveries. The same week, a Nature Machine Intelligence study put a number on why more agents don't automatically mean a better result: on some tasks, adding coordination costs 70 points instead of adding them.

A university lab takes the top spot

The South China Morning Post reports that as of July 21, Qiushi Engine, an agent built by a Zhejiang University-led team, held the top overall spot on ResearchClawBench, the Shanghai AI Laboratory-built benchmark that scores agents on whether they can carry out real end-to-end research and reach the same conclusions as published reference papers, or better ones. Open Science Desktop held #2 and Claude Code #3. We could not independently re-verify those exact standings: the live leaderboard renders client-side and its raw per-model scores weren't pullable at the time of writing, so treat the ranking as reported by SCMP rather than independently confirmed against raw numbers. The benchmark's own changelog does corroborate matching evaluation activity in the same window: a Qiushi run with GPT-5.5 logged July 15, an Open Science run with Claude Opus 4.8 logged July 9.

The underlying system is documented in "End-to-end autonomous scientific discovery on a real optical platform" (arXiv 2604.27092). In an open-ended run on physical lab hardware — not a simulation — the agent spent 145.9 million tokens across 3,242 LLM calls and 1,242 tool calls, producing 163 research notes and 44 scripts, to propose and experimentally validate "optical bilinear interaction," a physical mechanism the paper describes as structurally analogous to the core operation in Transformer attention. SCMP's reporting is explicit that the system "can't reliably make new discoveries yet" — this is one validated result out of an open-ended search, not a repeatable discovery pipeline.

The same week, a study explains when more agents actively hurts

A Nature Machine Intelligence paper published July 24 tested 260 configurations — six benchmarks crossed with five coordination architectures and three LLM families, holding prompts, tools, and compute fixed while varying only how agents were organized — and finds a "capability-saturation threshold": past a certain point, a stronger base model gets no further benefit from coordinating with other agents, and can get worse. A predictive model built from the single-agent baseline score alone calls the direction of the effect correctly in 94% of held-out configurations on SWE-bench Verified and Terminal-Bench. The measured swings run from +80.8% on structured financial-reasoning tasks to −70.0% on sequential-planning tasks, relative to a single agent working alone. The benchmarks here are general coding-and-reasoning tasks, not domain-science ones — but the mechanism is exactly the design question every agentic research tool has to answer: does this task actually benefit from more agents, or does one capable agent already have it covered?

What this means for reproducible, local-first science

Put the two stories together and the shared thread is cost, not capability. Qiushi Engine needed 3,242 LLM calls to reach one validated finding, from a system that, by its own benchmark's press coverage, still can't reliably make new discoveries. The Nature MI paper says that before you spend calls on coordinating a team of agents instead of running one, there is now a predictor — built from nothing more than the single-agent baseline — that gets the direction of the payoff right 94% of the time. For a workbench that runs its compute on your own machine, that tradeoff isn't abstract: every extra LLM call is wall-clock time and a bill you can watch accrue, which makes knowing in advance whether scaling up helps worth more than it is for a lab burning someone else's cloud budget.