Eight AI Models Score 80–98% on a Protein "Novelty" Test. So Does a Script That Never Trained on Anything, 110x Cheaper.

A new preprint put eight generative protein-structure models — RFDiffusion, Chroma, FrameDiff, FoldFlow, FrameFlow, ProtPardelle, BoltzGen, and PXDesign — through the same novelty check: cut a generated backbone into its structural domains, search each one against a database of known folds, and see how often at least one comes back matched. Every model scores 80.2% to 98.2%. Then the authors ran the identical check on a script they built with zero training — retrieve three domain fragments from a reference library, glue them together with idealized helical linkers, done — and it scored 96.0%, in 1.98 seconds per backbone on a CPU, about 110 times faster than their own timed RFDiffusion run. On the metric most protein-design papers report by default, five years of diffusion and flow-matching research is statistically tied with an afternoon of retrieval code.

That is not a takedown of the eight models — three of them, including RFDiffusion itself, clearly beat the zero-training baseline on the metric most novelty claims actually lean on, whether the entire generated chain matches something whole in the database. The real finding is narrower and more useful than a gotcha: the metric the field defaults to rewards a design for not resembling one whole known protein, while saying almost nothing about whether it was built out of known parts. A script that only knows how to cut and paste clears that bar almost as well as five years of diffusion and flow-matching research.

What the novelty metric is actually checking

"Is Retrieval All You Need? Assessment and Emergence of Novelty in Protein Structure Generation" (Xu, Zhang, He, Shen, Liu, Ying, Tan; arXiv:2608.10598, posted August 11, 2026) introduces the Domain Retrieval Rate (DRR): decompose each generated backbone into its structural domains with Merizo, search each domain independently against CATH S40, a curated reference set of 34,653 non-redundant protein domains, and score the backbone as domain-retrievable if at least one piece matches at alnTM≥0.5, the paper's "same-fold" threshold. This is a local, per-domain check, distinct from full-chain retrievability, which asks whether the entire generated chain matches one whole known structure at qTM≥0.5. The team ran 500 backbones through each of eight generators — 100 at each of five lengths from 100 to 500 residues — and found DRR held at 80.2% to 98.2% across the board. Full-chain retrievability, by contrast, ranged from as low as 0.6% for four of the models up to 75.4% for PXDesign. Read together, those two numbers say the same thing every time: a generated protein rarely matches one whole known structure, but it is still, almost always, assembled from pieces that do.

Why the distinction matters: a drug-discovery or enzyme-design team doesn't care whether a generated backbone matches something in a public database in the abstract, they care whether the model can produce a genuinely new scaffold when no known one does the job — a new binding pocket geometry, a new catalytic arrangement, something CATH S40's 34,653 domains don't already contain. Full-chain retrievability is the metric that's supposed to answer that question, and it does vary meaningfully across the eight models, from PXDesign's 75.4% down to a shared ceiling of 0.6% or below for four others. But almost every paper in this space leads with novelty language pointed at the wrong number, because domain-level reuse doesn't show up in a rendered structure the way full-chain mismatch does — a backbone built from three known domains in a new arrangement looks, to the eye and to a quick TM-align check against a handful of reference structures, exactly as novel as one that isn't.

A script that never trained on anything

The paper's sharpest instrument is RetFold, a baseline built with no gradient-based learning at all. It retrieves three CATH S40 fragments for a target length — one core fragment sized 40–60% of the target, plus two extension fragments at 15–35% each — ranked for compatibility using an ensemble of ESM-2, Foldseek 3Di, and ProstT5 embeddings and filtered by CATH hierarchy distance. It then replaces the joins between fragments with idealized α-helical linkers, sampling linker length and rotational phase, filtering survivors for steric clash and compactness, and validating what is left with ProteinMPNN and AlphaFold3. Nothing about that process involves a trained generative model. Run on a CPU alone, it produces a backbone in 1.98 seconds, versus roughly 218 seconds for a timed RFDiffusion run on the same hardware class — about 110 times faster — and its resulting Domain Retrieval Rate, 96.0%, and full-chain retrievability, 20.0%, land inside the spread the eight learned models produce. On the metric most papers in this space report, a retrieval-and-glue script that took an afternoon to write is indistinguishable from state-of-the-art generative modeling.

The honest tell

The paper does not claim RetFold is a good protein designer, and it has the evidence to prove it isn't: the mean pLDDT of RetFold's own fragment junctions is 43.1, against 79.1 for the borrowed fragments themselves before they were stitched together. Those junctions are structurally weak by construction — idealized helices sampled for geometric fit, not designed for foldability — which is exactly the point. A method that scores well on the field's default novelty metric while producing backbones that would fail a real structural check is evidence about what the metric measures, not a claim that cut-and-paste protein design works. The authors are careful not to overreach in the other direction either: a stricter, reference-normalized criterion (tTM≥0.7) does reliably separate RetFold from the real generators, and an aligned-length score on its own still retrieves 90.04% of queries whose entire fold class was deliberately removed from the reference set, meaning even the domain-level check carries a real false-positive rate that a careful reader has to hold in mind. The finding isn't "novelty metrics are fake." It is narrower and more actionable: the specific, most commonly reported one is easy to match without learning anything, and a paper that doesn't report what a trivial retrieval baseline scores on its own metric is skipping the control that would tell you whether its capability claim survives contact with one.

What this means for reproducible, local-first science

None of this is a claim about where any particular commercial protein-design tool stands — MegaBrain BioScience doesn't generate protein backbones, and nothing here says RFDiffusion or PXDesign are worse than their headline numbers suggest; three of the eight models in this study clearly do outperform a retrieval baseline on the metric that matters most. What the paper adds to this beat's running argument is a specific, checkable question to ask of the next "AI designs a novel protein" claim: which novelty metric is being reported, and has anyone run the trivial retrieval baseline against it? That is not a question you can answer from a press release or an abstract — it requires rerunning the comparison yourself, against the same reference database, on the same backbones. A workbench that exports a reproducibility record for every run doesn't settle whether a given design is genuinely novel. It does mean that when a novelty number like this one shows up, the domain segmentation, the reference database, and the retrieval baseline behind it are things you can go check yourself, instead of a claim you take on the strength of a headline. That is the same standard this beat has applied to capability claims all year, from a phage-design hit rate to a diagnostic accuracy figure to a benchmark pass rate — the number in the headline is real, and the honest version of it is almost always narrower, cheaper to fake, or both.