An AI Biology-Discovery Agent Nailed the Fit Test. Its Plausibility Score Still Collapsed From 0.98 to 0.59.

A new AI agent for discovering the equations behind biological systems nailed its hardest fit test in a new paper, matching the training trajectory closely. Its plausibility score collapsed anyway, from 0.98 to 0.59. The paper, from Weizmann Institute researchers building an LLM-and-symbolic-regression agent to model biological dynamics, isolates exactly why: strip out the mechanistic constraints and domain knowledge, and what's left explains the data well and the biology badly. Two more papers published the same week land on the identical failure from completely different directions — one auditing a totally unrelated task, one arguing from first principles that today's predictive AI-for-science paradigm has this gap built in by design.

A model that fits the data and gets the biology wrong

"Automatic Ordinary Differential Equations Discovery For Biological Systems Using Large Language Model Powered Agentic System" (arXiv 2607.13608, Krongauz, Zulti, Segal & Lazebnik, submitted July 15) introduces MEDA, an agent that retrieves background knowledge, defines admissible variables, generates mechanistic constraints, proposes candidate ODEs, and fits and evaluates them — a pipeline aimed squarely at the difference the paper draws in its own abstract, between "data-fitting tools" and "mechanistic models of the universe." The authors test it across 12 biological and biologically-inspired systems in three tiers: 5 canonical systems for retrieval (logistic growth, Lotka–Volterra, SIR, the Allee effect, FitzHugh–Nagumo), 5 for reasoning-based extrapolation to unseen variants, and 2 for open-ended discovery with real experimental data — misinformation-spread dynamics and chronic-wound progression.

The result that matters isn't the headline accuracy number. It's the ablation. MEDA's full system, with literature retrieval and mechanistic constraints intact, scores a term-level F1 of 1.00 and a plausibility score of 0.967, averaged across the evaluation set. Remove the literature grounding and keep only data-driven formalization, and both numbers fall — F1 to 0.732, plausibility to 0.700. Strip everything down to a pure symbolic-regression baseline (SINDy, with no literature, no formalization, no constraints — the closest thing in the paper to "just fit the data") and F1 drops to 0.541, plausibility to 0.603, with term precision falling to 0.382. The paper's own read: SINDy "often recovered dominant driver terms, but introduced many false-positive terms, especially in conserved or coupled equations."

The sharpest single data point sits inside that no-literature condition, on the misinformation-dynamics system: plausibility drops from 0.98 to 0.59 and term-level F1 from 1.00 to 0.50 — and the paper is explicit that this happens "despite good trajectory fitting." In plain terms, the equation still tracked the observed curve. It just wasn't the right equation. The authors' own conclusion states the finding as cleanly as it can be stated: "knowledge-guided formalization and mechanistic constraints are load-bearing components, whereas numerical fitting alone can preserve trajectory-compatible but biologically incorrect equations."

The same failure, in a completely different domain

"Analogical Deep Research: Retrieving and Integrating Historical Analogies for Foresight Analysis" (arXiv 2607.13602, Chen, Chen, Sun & Zhang, submitted July 15) has nothing to do with biology or differential equations. It asks whether deep-research agents can find historically analogous events to reason about an unfolding situation — the kind of forecasting task an intelligence analyst or policy researcher does by hand. The authors built the first benchmark for it, ADR-bench: 15 events, 10 with literature-documented historical analogies and 5 recent ones without, tested against 10 agent configurations including OpenAI Deep Research, Gemini Deep Research, Qwen Deep Research, and three vanilla agents running Claude Sonnet 4.5, GPT-5.4, and Qwen3-8B.

Their diagnosis of why the existing agents struggle reads almost like a restatement of MEDA's ablation in a different vocabulary: "LLM agents are poor at finding analogies because they match on surface features rather than underlying mechanisms." The fix the paper proposes, CANA (Causal Analogical Researcher), works by forcing mechanism alignment and cross-analogy confirmation instead of surface similarity — and the headline result is that this brings up to 10% improvement in historical-analogy generation over the prior best method, enough to beat every deep research agent tested on ADR-bench. (A separate, narrower comparison in the paper's fine-grained rubric evaluation reports gains as high as 25% for one specific model/metric combination; the 10% figure is the one that generalizes across the benchmark and the one the paper leads with.) Two unrelated teams, working on two unrelated problems — ODE discovery in biology and analogical reasoning in forecasting — independently found that the thing separating a model that looks right from a model that is right is whether it's reasoning about mechanism or pattern-matching on surface structure.

What the theorists say the models are actually missing

A third paper published the same week gives that pattern a name and an argument for why it's structural, not incidental. "From Observation to Insight: Mechanistic World Models and the Quest for Autonomous Discovery" (arXiv 2607.12474, Ingmar Posner and Anson Lei of Oxford's Applied AI Lab, and Bernhard Schölkopf of the Max Planck Institute for Intelligent Systems and the ELLIS Institute Tübingen, submitted July 14) opens with a sentence that could serve as this entire week's thesis statement: "prediction alone does not constitute scientific discovery." Their argument is that contemporary foundation models, however accurate at prediction, "remain fundamentally organised around predictive mappings rather than explanatory structure" — and they back it with three concrete examples from published work, not speculation.

Protein language models like ESM-2 have sparse autoencoders that recover interpretable features corresponding to real binding sites and structural motifs — a genuine partial win for the mechanistic view. Single-cell foundation models like Geneformer and scGPT fare worse: the paper cites a systematic investigation of their attention patterns finding "computation circuits that reflect statistical co-expression rather than the underlying causal pathways of gene regulation," and notes these models "do not yet outperform simpler baselines in perturbation prediction tasks." The starkest example is a world model trained on planetary orbits: Posner, Lei, and Schölkopf cite work by Vafa et al. showing that a model trained purely to predict planetary trajectories "consistently fail[s] to internalise Newtonian mechanics despite being able to accurately predict planet trajectories." A model can nail the physics homework and still not know physics.

What this means for reproducible, local-first science

None of these three papers name or test any commercial research product, MegaBrain BioScience included — this isn't a claim about who's ahead. What they add up to is a more specific version of a warning this blog keeps returning to: a fit score, an accuracy number, or a benchmark rank tells you whether an agent's output resembles the right answer, not whether it got there for the right reason. MEDA's own ablation is the cleanest demonstration available — two configurations can post near-identical trajectory fit while one of them is quietly wrong about the biology, and the only way to tell the difference is to check the mechanism, not just the curve.

That's a harder thing to check than an accuracy score, and it's exactly the kind of check that only works if you can see the agent's reasoning, re-run its steps, and inspect the constraints it applied — not just read off a final number. A workbench that exports a reproducibility record for every result, and pairs that with an independent review step, doesn't make an agent more likely to find the right mechanism. It makes it possible to catch the case, like MEDA's misinformation-dynamics run, where the fit looked fine and the science wasn't.