This paper-specific evaluation tests Zero-shot substitution mutation effects: stability using ProteinGym substitution DMS: stability assays.
Explanatory profile: limited source coverage · Automated source review, 2026-09-16. This does not change the review status of its results.
| Property | Description and evidence |
|---|---|
| Record type | Paper-specific task; protocol incompletely extractedProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Table A7, ESM-2 (15B) row, Stability column; Table A7, ProteinMPNN row, Stability column |
| Inputs | ProteinGym substitution DMS: stability assaysProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Table A7, ESM-2 (15B) row, Stability column; Table A7, ProteinMPNN row, Stability column |
| Assessment | Mean Spearman rhoProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Table A7, ESM-2 (15B) row, Stability column; Table A7, ProteinMPNN row, Stability column |
| Recorded split or evaluation setting | UnextractedProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Table A7, ESM-2 (15B) row, Stability column; Table A7, ProteinMPNN row, Stability column |
| Datasets | Not extracted or verified for this record. |
| Organisms | Not extracted or verified for this record. |
| Assays | Not extracted or verified for this record. |
| Adaptation | Not extracted or verified for this record. |
| Baselines | Not extracted or verified for this record. |
Outline of the existing paper extraction. Split membership, fitting details and scorer implementation remain incompletely reviewed.
The existing paper extraction describes: Zero-shot mutation scores; average Spearman across stability-category assays. This description is retained with the exact evaluation records; it is not a new protocol reconstruction.
ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Table A7, ESM-2 (15B) row, Stability column; Table A7, ProteinMPNN row, Stability columnRelease 2026-09-16-d74d282221a9 · 2 evaluations · 2 metric rows. Different protocols are not a single leaderboard.
| Metric and finding | Coverage and uncertainty | Evidence |
|---|---|---|
| ESM-2: Zero-shot substitution mutation effects: stability Model: ESM-2 · Benchmark: Zero-shot substitution mutation effects: stability · Dataset: ProteinGym substitution DMS: stability assays Zero-shot mutation scores; average Spearman across stability-category assays. Independent external evaluation · Evaluation metadata: needs review | ||
| 0.488 Mean Spearman rho Unit: unitless · Direction: unknown | Uncertainty: not reported in legacy extract Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Table A7, ESM-2 (15B) row, Stability column Source checking is not independent reproduction. |
| ProteinMPNN: Zero-shot substitution mutation effects: stability Model: ProteinMPNN · Benchmark: Zero-shot substitution mutation effects: stability · Dataset: ProteinGym substitution DMS: stability assays Zero-shot mutation scores; average Spearman across stability-category assays. Independent external evaluation · Evaluation metadata: needs review | ||
| 0.566 Mean Spearman rho Unit: unitless · Direction: unknown | Uncertainty: not reported in legacy extract Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Table A7, ProteinMPNN row, Stability column Source checking is not independent reproduction. |
Catalogue extraction inspected; protocol claims remain limited to the cited evidence. Missing details are not presumed absent from the original paper.
Stable record: reported-task-6243658a1bc215Release 2026-09-16-d74d282221a9 · Record review: needs review
Stable ID: reported-task-6243658a1bc215