A Protein-Dynamics Benchmark Scored 0.963 on Scrambled Data. The Real Number Is 0.004.

A new preprint took the frames from a real molecular-dynamics simulation, shuffled their order into physical nonsense, and re-ran the standard test used to certify that an AI protein-dynamics generator has "learned dynamics." The score barely moved. A second test, built to check time-ordering directly, collapsed from 0.963 to 0.004. Below: that result, plus a clinical coding agent whose accuracy nearly halved between two hospitals in one city, a single-cell drug-response model, an Alzheimer's ablation, and a coding agent turned loose on a real hospital data warehouse.

Structural biology: a benchmark stayed high after its input was ruined

Protein "dynamics" generators are increasingly used as a faster substitute for molecular dynamics (MD) simulation, and the field's standard way to validate one is an ensemble-fidelity test: does the set of generated conformations match a reference equilibrium ensemble? A team at Zhejiang University asked whether that test can tell the difference between a model that has learned how a protein moves over time and one that has only learned which shapes it visits. They took authentic MD trajectory frames and randomly reordered them — destroying every trace of real time-ordered dynamics while leaving the exact same set of conformations intact. Ensemble fidelity was unchanged. Their new benchmark, Dynbench, which separately gates on time-dependent, trajectory-admissible behavior, collapsed from a 0.963 pass rate to 0.004; an independent real MD replicate, run through the identical test, still scored 0.95. In a prospective 36-protein lockbox test, generators that looked similar under ensemble fidelity separated sharply under Dynbench, and two of them swapped rank order entirely. The paper is a preprint, posted September 4, and not yet peer-reviewed — but the logic of the reordering test doesn't depend on peer review to check: a metric that survives having its dynamics deliberately destroyed was never measuring dynamics. Liu, Qian & Chi, bioRxiv 10.64898/2026.09.02.748111.

Clinical & translational AI: same coding agent, two hospitals, a 32-point accuracy gap

The full result is today's feature: 83.6% at the Hospital That Trained It. 52.0% at a Different One. Same AI Coding Agent. The short version: Clinico, an AI agent built at Zhongshan Hospital, Fudan University to adjudicate ICD diagnosis codes across an admission, hit 83.6% principal-diagnosis exact-match agreement and 90.6% complete-code-set micro-F1 on 5,000 internal test stays — beating every comparator, including two systems trained on 80,340 stays. Tested on 4,109 stays from a second Shanghai hospital it had never seen, those numbers fell to 52.0% and 42.6%. Every comparator system fell too. Clinico still finished first at both sites; it just was not the same system at the second one.

Single-cell: a knowledge graph helps predict how cells respond to untested drugs

Experimentally profiled drug-perturbation data covers only a sliver of chemical space, and models built to predict single-cell response to an unprofiled compound typically treat drugs as bare identifiers, with no way to reason about a new one. A Shanghai Jiao Tong University team's MAP framework instead builds MAP-KG, a knowledge graph unifying 14 public resources across 187,000 drugs, 23,000 genes, and 694,000 mechanistic relationships, and pretrains mechanism-aware compound and gene embeddings from it via contrastive learning. Coupled to a pretrained single-cell foundation model, MAP improves top-50 differentially-expressed-gene Pearson correlation by up to 13.3% for unseen cell-type-drug combinations and 12.2% for the stricter case of entirely unprofiled drugs, over the strongest baselines across three benchmarks. In an in-silico screen of the non-small-cell lung cancer line A-549, MAP's pathway-level predictions ranked 4 of 5 approved anti-cancer drugs among its top candidates — a retrospective consistency check against known activity, not a new wet-lab result. Posted to bioRxiv in February, published in Nature Machine Intelligence on August 26.

HealthTech: an ablation finds one biomarker family carries the Alzheimer's signal

Which biomarker predicts progression from mild cognitive impairment (MCI) to Alzheimer's disease has stayed unclear even as multimodal models have multiplied. A systematic ablation across 2,430 ADNI participants with known 24-month outcomes found the full multimodal model — demographics, cognition, APOE genotype, structural MRI, cerebrospinal fluid, and PET — reached an AUC of 0.934. Removing cognitive assessments alone dropped that to 0.883, the largest single-modality loss by far; pulling APOE, CSF, or MRI cost only 0.001–0.003 AUC each. PET's own contribution may be underestimated: a complete-case sensitivity analysis found that imputing missing PET values had significantly inflated its apparent contribution (P=0.005) in the main analysis. A demographics-only baseline model scored 0.556, barely above chance. The study is single-author work from a commercial AI lab, not yet peer-reviewed — a real limitation on a clean result. medRxiv 10.64898/2026.09.01.26360413.

HealthTech: a general-purpose coding agent got most answers right, reproducibly got few

Health systems typically answer data questions by having an analyst hand-write queries against the electronic health record warehouse. A Stanford team tested whether OpenAI Codex, given read-only access to a full copy of a production warehouse (Epic Caboodle), could do the same for ten questions about a common pediatric infection, acute otitis media. Working autonomously, the agent never fabricated data and landed within 5% of an analyst-adjudicated reference on 27 of 30 runs — but matched it exactly on only 10, and three repeated runs on the same question returned an identical answer just 3 times out of 10. The failure mode was not the agent's reasoning: it was warehouse knowledge that the data dictionary omits. Once analysts supplied a corrected definition for one concept, reproducibility and accuracy both recovered on related questions. The authors' own framing: a capable drafting aid under supervision, not yet an autonomous analyst. medRxiv 10.64898/2026.09.02.26362008.