Task

ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall

Zero-shot substitutions, top 10% recall. Scored with Top 10% recall on ProteinGym substitution DMS assays. Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes.

21 evaluations · 21 metric rows

Overview

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

Evaluation design

Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.

Benchmarks

ProteinGym

These source-backed links do not make different protocols or scores interchangeable.

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Explore all linked results

Run instructions

No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.

Published comparisons

Explore the results reported under one evaluation protocol. Each figure keeps its source, dataset and metric together; it is not a ranking across studies. The pooled view gathers every source table that reports the same metric and names what it does not hold constant.

Comparison and metric

ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall

recall (fraction) · Higher values are better for this metric.

Every method ProteinGym reports on Zero-shot substitutions, top 10% recall, scored with Top 10% recall on ProteinGym substitution DMS assays.

Evaluation protocol · ProteinGym substitution DMS assays (ProteinGym split)

Site-Independent · Configuration · Author-reported evaluation0.201
WaveNet · Configuration · Author-reported evaluation0.203
EVmutation · Configuration · Author-reported evaluation0.222
DeepSequence (ensemble) · Configuration · Author-reported evaluation0.226
EVE (ensemble) · Configuration · Author-reported evaluation0.230
GEMME · Configuration · Author-reported evaluation0.211
UniRep · Configuration · Author-reported evaluation0.139
CARP (640M) · Configuration · Author-reported evaluation0.208
ESM-1b · Configuration · Author-reported evaluation0.203
ESM-2 (15B) · Configuration · Author-reported evaluation0.208
RITA XL · Configuration · Author-reported evaluation0.193
ESM-1v (ensemble) · Configuration · Author-reported evaluation0.211
ProGen2 XL · Configuration · Author-reported evaluation0.199
VESPA · Configuration · Author-reported evaluation0.201
UniRep evotuned · Configuration · Author-reported evaluation0.181
MSA Transformer (ensemble) · Configuration · Author-reported evaluation0.224
Tranception L · Configuration · Author-reported evaluation0.220
TranceptEVE L · Configuration · Author-reported evaluation0.230
ESM-IF1 · Configuration · Author-reported evaluation0.223
MIF-ST · Configuration · Author-reported evaluation0.226
ProteinMPNN · Configuration · Author-reported evaluation0.186

Source order is preserved. Plotted marks show point estimates; uncertainty, where reported, is retained in the printed values and table. Differences do not establish statistical significance.

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, column(Zero-shot substitutions, top 10% recall)

Values, uncertainty and evidence

recall: original source values
Tested entity	Printed value	Uncertainty	Evidence
Site-Independent · Configuration	0.201 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(Alignment- Site-Independent), column(Zero-shot substitutions, top 10% recall)
WaveNet · Configuration	0.203 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(based WaveNet), column(Zero-shot substitutions, top 10% recall)
EVmutation · Configuration	0.222 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(EVmutation), column(Zero-shot substitutions, top 10% recall)
DeepSequence (ensemble) · Configuration	0.226 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(DeepSequence (ensemble)), column(Zero-shot substitutions, top 10% recall)
EVE (ensemble) · Configuration	0.230 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(EVE (ensemble)), column(Zero-shot substitutions, top 10% recall)
GEMME · Configuration	0.211 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(GEMME), column(Zero-shot substitutions, top 10% recall)
UniRep · Configuration	0.139 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(Protein UniRep), column(Zero-shot substitutions, top 10% recall)
CARP (640M) · Configuration	0.208 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(language CARP (640M)), column(Zero-shot substitutions, top 10% recall)
ESM-1b · Configuration	0.203 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-1b), column(Zero-shot substitutions, top 10% recall)
ESM-2 (15B) · Configuration	0.208 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-2 (15B)), column(Zero-shot substitutions, top 10% recall)
RITA XL · Configuration	0.193 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(RITA XL), column(Zero-shot substitutions, top 10% recall)
ESM-1v (ensemble) · Configuration	0.211 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-1v (ensemble)), column(Zero-shot substitutions, top 10% recall)
ProGen2 XL · Configuration	0.199 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ProGen2 XL), column(Zero-shot substitutions, top 10% recall)
VESPA · Configuration	0.201 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(VESPA), column(Zero-shot substitutions, top 10% recall)
UniRep evotuned · Configuration	0.181 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(Hybrid UniRep evotuned), column(Zero-shot substitutions, top 10% recall)
MSA Transformer (ensemble) · Configuration	0.224 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(MSA Transformer (ensemble)), column(Zero-shot substitutions, top 10% recall)
Tranception L · Configuration	0.220 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(Tranception L), column(Zero-shot substitutions, top 10% recall)
TranceptEVE L · Configuration	0.230 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(TranceptEVE L), column(Zero-shot substitutions, top 10% recall)
ESM-IF1 · Configuration	0.223 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(Inverse ESM-IF1), column(Zero-shot substitutions, top 10% recall)
MIF-ST · Configuration	0.226 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(Folding MIF-ST), column(Zero-shot substitutions, top 10% recall)
ProteinMPNN · Configuration	0.186 fraction	Not reported	Author-reported evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ProteinMPNN), column(Zero-shot substitutions, top 10% recall)

Scope and limitations

Author-reported numbers, source checked but not independently reproduced.
MSE is better when lower; every other metric here is better when higher.
Zero-shot, supervised and indel benchmarks use different protocols and assay sets, so their figures are not comparable to each other.

Source transcription and grouping reviewed by automated source review on 2026-09-18. These experiments were not independently reproduced by rewire.

Tested entities and results

Release 2026-09-17-134cd1815de8 · 21 evaluations · 21 metric rows. Different protocols are not a single leaderboard. Where several source tables report the same metric, the published comparisons above offer a pooled view that names what it does not hold constant.

Evaluation setupMetricEvidence origin

Results grouped by the exact reported evaluation
Metric and finding	Coverage and uncertainty	Evidence
CARP (640M) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: CARP (640M)Task: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.208 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(language CARP (640M)), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
DeepSequence (ensemble) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: DeepSequence (ensemble)Task: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.226 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(DeepSequence (ensemble)), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
ESM-1b on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: ESM-1bTask: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.203 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-1b), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
ESM-1v (ensemble) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: ESM-1v (ensemble)Task: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.211 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-1v (ensemble)), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
ESM-2 (15B) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: ESM-2 (15B)Task: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.208 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-2 (15B)), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
ESM-IF1 on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: ESM-IF1Task: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.223 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(Inverse ESM-IF1), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
EVE (ensemble) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: EVE (ensemble)Task: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.230 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(EVE (ensemble)), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
EVmutation on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: EVmutationTask: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.222 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(EVmutation), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
GEMME on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: GEMMETask: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.211 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(GEMME), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
MIF-ST on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: MIF-STTask: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.226 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(Folding MIF-ST), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
MSA Transformer (ensemble) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: MSA Transformer (ensemble)Task: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.224 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(MSA Transformer (ensemble)), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
ProGen2 XL on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: ProGen2 XLTask: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.199 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ProGen2 XL), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
ProteinMPNN on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: ProteinMPNNTask: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.186 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ProteinMPNN), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
RITA XL on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: RITA XLTask: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.193 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(RITA XL), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
Site-Independent on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: Site-IndependentTask: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.201 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(Alignment- Site-Independent), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
TranceptEVE L on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: TranceptEVE LTask: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.230 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(TranceptEVE L), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
Tranception L on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: Tranception LTask: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.220 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(Tranception L), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
UniRep evotuned on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: UniRep evotunedTask: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.181 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(Hybrid UniRep evotuned), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
UniRep on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: UniRepTask: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.139 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(Protein UniRep), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
VESPA on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: VESPATask: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.201 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(VESPA), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.
WaveNet on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: WaveNetTask: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked
0.203 recall Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(based WaveNet), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

1 evidence row matching the loaded filters

Claims, original sources and review scope · Release 2026-09-17-134cd1815de8
Property and statement	Original source and location	Review and provenance
Relationship: part of discovery-benchmark-proteingym Individual claims	ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design Original source ↗ Table 2, column(Zero-shot substitutions, top 10% recall) Version: Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256 Retrieved: 2026-09-17T08:06:28.522924+00:00	source checked automated source review · 2026-09-18 Audit details Primary-source transcription with no human sign-off and no independent reproduction. Field: `links:part_of:discovery-benchmark-proteingym` Claim: proteingym-association-zs-sub-recall Source artifact SHA-256: `a3b08cc4a6befd64620cf0f287d78d55a36dc2639955f52c5655f83833c50104` Hash scope: Exact retrieved primary paper artifact bytes. Inspected artifact

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

Release 2026-09-17-134cd1815de8 · Record review: source checked

1 source records and release history

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Original source · Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256

Download this release

Technical metadata and extraction receipts

Stable ID: proteingym-task-zs-sub-recall

areas: proteins-complexes
tasks: Zero-shot substitutions, top 10% recall
metric: Top 10% recall
metric direction: higher
dataset: ProteinGym substitution DMS assays
protocol: Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes.
source locator: Table 2, column(Zero-shot substitutions, top 10% recall)
comparison panels: id: proteingym-panel-zs-sub-recall; title: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall; protocol id: proteingym-task-zs-sub-recall; dataset id: proteingym-dataset-proteingym-substitution-dms-assays; metric: recall; unit: fraction; direction: higher; result ids: proteingym-result-site-independent-zs-sub-recall-recall; proteingym-result-wavenet-zs-sub-recall-recall; proteingym-result-evmutation-zs-sub-recall-recall; proteingym-result-deepsequence-ensemble-zs-sub-recall-recall; proteingym-result-eve-ensemble-zs-sub-recall-recall; proteingym-result-gemme-zs-sub-recall-recall; proteingym-result-unirep-zs-sub-recall-recall; proteingym-result-carp-640m-zs-sub-recall-recall; proteingym-result-esm-1b-zs-sub-recall-recall; proteingym-result-esm-2-15b-zs-sub-recall-recall; proteingym-result-rita-xl-zs-sub-recall-recall; proteingym-result-esm-1v-ensemble-zs-sub-recall-recall; proteingym-result-progen2-xl-zs-sub-recall-recall; proteingym-result-vespa-zs-sub-recall-recall; proteingym-result-unirep-evotuned-zs-sub-recall-recall; proteingym-result-msa-transformer-ensemble-zs-sub-recall-recall; proteingym-result-tranception-l-zs-sub-recall-recall; proteingym-result-trancepteve-l-zs-sub-recall-recall; proteingym-result-esm-if1-zs-sub-recall-recall; proteingym-result-mif-st-zs-sub-recall-recall; proteingym-result-proteinmpnn-zs-sub-recall-recall; source ids: evidence-expansion-proteingym-a3b08cc4; source locator: Table 2, column(Zero-shot substitutions, top 10% recall); context: Every method ProteinGym reports on Zero-shot substitutions, top 10% recall, scored with Top 10% recall on ProteinGym substitution DMS assays.; caveats: Author-reported numbers, source checked but not independently reproduced.; MSE is better when lower; every other metric here is better when higher.; Zero-shot, supervised and indel benchmarks use different protocols and assay sets, so their figures are not comparable to each other.; review: method: automated_source_review; date: 2026-09-18

Related records

Suggest a correction