Claude Opus 4.7 Ranks #1 on the AI Science Leaderboard. It Scores 21 Out of 100 on the Benchmark That Grades the Work.

Claude Opus 4.7 tops AstaBench, the leaderboard the field uses to rank AI research agents, at 58.0%. Put the same model class through ResearchClawBench, a benchmark that grades an agent on whether it can reproduce the real published paper it was assigned, and the best score belongs to Claude Code: 21.5 out of 100. That 37-point gap isn't a fluke of one hard test. It's a pattern, and this week's other release shows exactly where the real capability is hiding: not in bigger, more general models, but in narrow ones built for a single job.

The leaderboard everyone quotes, and the one nobody does

AstaBench, Ai2's open benchmark for AI research agents, is the number that gets cited in press releases: "our model leads AstaBench." Its spring 2026 update (April 30) has Claude Opus 4.7 on top at 58.0% overall, ahead of Claude Opus 4.6 (55.3%), Claude Sonnet 4.6 (54.5%), and GPT-5.5 (52.9%, at less than half the per-problem cost). It's built from over 2,400 problems spanning literature search, code execution, data analysis, and end-to-end discovery tasks — a broad, defensible instrument, and genuinely useful for tracking progress across labs.

But an aggregate score is an average, and averages hide their hardest components. Literature search and code execution are tasks today's frontier models are already good at; blending them into the same number as full-cycle discovery lets a strong score on the easy 80% carry a weak score on the hard 20%. AstaBench doesn't hide this — its own category breakdown shows exactly where scores drop. The problem is which number gets quoted afterward.

ResearchClawBench: grade the agent against the paper it's copying

ResearchClawBench(arXiv 2606.07591, revised to v5 on July 3) is built to remove that averaging effect entirely. It takes 40 tasks across 10 scientific domains, each one grounded in a real published paper, and asks an agent to go from data to code to a publication-quality report — then grades the output against what the paper actually found, not against a rubric an LLM judge approximates. The authors ran seven autonomous research agents and seventeen standalone LLMs through the same protocol. The best autonomous agent, Claude Code, averaged 21.5 out of 100. The best standalone LLM, Claude Opus 4.7 — the same model leading AstaBench — averaged 20.7. The paper's own conclusion: "current systems remain far from reliable re-discovery."

That's not a knock on AstaBench's methodology; it's a demonstration of what happens when a benchmark stops averaging across difficulty levels and instead isolates the single hardest thing a research agent is supposed to do: reproduce a result well enough that it matches the ground truth, unassisted, end to end. The gap between 58.0% and roughly 21% is the gap between "good at research-shaped tasks" and "good at research."

The counterexample: a narrow agent that isn't just better, it's a different animal

The same week produced the sharpest evidence yet for what actually closes that gap. DeepEvidence, published in Nature Machine Intelligence this month (preprint: arXiv 2601.11560), is not a generalist research agent at all — it's a two-part orchestrator purpose-built for one job: exploring and synthesizing evidence across biomedical knowledge graphs and literature, via a breadth-first search agent and a depth-first reasoning agent working together.

Run against the same class of general-purpose models AstaBench and ResearchClawBench test, DeepEvidence doesn't win narrowly. On HLE-Medicine, it scores 40.0% against Biomni's 20.0%, ToolUniverse's 10.0%, and 3.3% for both Claude Sonnet 4.5 and GPT-5 — a 12x margin over two frontier generalist models on the same test. On LabBench-LitQA2, it hits 80.0% against Biomni's 32.0% and Sonnet-4.5's 48.0%. On SuperGPQA-Medicine-Hard, a harder gap-closing margin: 47.1% against Sonnet-4.5's 43.6%. On TrialPanorama-EvidenceQA, 96.0% against Sonnet-4.5's 88.0%. Every comparison in the paper goes the same direction.

Biomni itself is worth noting here, not as a loser but as a data point about the limits of general-purpose breadth: it's a credible, peer-reviewed biomedical agent in its own right, and it still loses to DeepEvidence by roughly 2x on HLE-Medicine and LitQA2. The pattern isn't "narrow beats broad by a little." It's "narrow, purpose-built architecture for a specific evidence task beats every general-purpose alternative by a margin no amount of extra parameters closed."

Why this isn't the same argument as last week

It would be easy to fold this into the ground this newsletter has already covered: model size isn't the bottleneck, and neither is raw agent count. This is a third, narrower claim, and the data supports it specifically. The bottleneck isn't how big the model is or how many copies you run — it's whether the system was built for the exact task in front of it. A generalist frontier model, however capable on broad benchmarks, is being asked by ResearchClawBench and DeepEvidence's baselines to do something it wasn't architected for: track evidence across a structured graph, or reproduce a specific paper's exact experimental protocol. A system built around that one task, even with a fraction of the general capability, wins by a landslide.

That has a direct, unglamorous implication for how the field should read its own scoreboard. A rising AstaBench number is real progress on the tasks it measures, but it is not evidence that autonomous, end-to-end scientific reproduction is close. Two different benchmarks, two different domains, both point the same direction: the capability that's actually shipping right now is narrow and specific, not general and scaled.

What this means for reproducible, local-first science

If the field's real progress is coming from narrow, purpose-built systems rather than ever-larger generalists, the practical lesson isn't "wait for a bigger model." It's that the tooling around a research agent — the domain skills, the evidence tracking, the verification step that checks output against a ground truth rather than an LLM's own confidence — is doing more of the real work than the base model gets credit for. DeepEvidence didn't beat Sonnet-4.5 with a bigger model; it beat it with an architecture built around evidence attribution. That's the same principle behind exporting a reproducibility record for every result, not trusting a single aggregate score to tell you whether a result actually holds up.