result · source checked
0.488 Mean Spearman rho
ESM-2 · Mean Spearman rho · ProteinGym substitution DMS: stability assays
- Tested model
- ESM-2
- Task or benchmark
- Zero-shot substitution mutation effects: stability
- Dataset
- ProteinGym substitution DMS: stability assays
- Procedure
- Zero-shot mutation scores; average Spearman across stability-category assays.
- Evaluation
- ESM-2: Zero-shot substitution mutation effects: stability
- Evidence
- Independent external evaluation · source checkedProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Table A7, ESM-2 (15B) row, Stability column
A source-checked result verifies the numerical transcription, not every model or protocol detail. Evaluation metadata: needs review. Source checked does not mean independently reproduced.
Evaluation results
Release 2026-09-16-d74d282221a9 · 1 evaluation · 1 metric row. Different protocols are not a single leaderboard.
| Metric and finding | Coverage and uncertainty | Evidence |
|---|---|---|
| ESM-2: Zero-shot substitution mutation effects: stability Model: ESM-2 · Benchmark: Zero-shot substitution mutation effects: stability · Dataset: ProteinGym substitution DMS: stability assays Zero-shot mutation scores; average Spearman across stability-category assays. Independent external evaluation · Evaluation metadata: needs review | ||
| 0.488 Mean Spearman rho Unit: unitless · Direction: unknown | Uncertainty: not reported in legacy extract Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Table A7, ESM-2 (15B) row, Stability column Source checking is not independent reproduction. |
Sources and history
Release 2026-09-16-d74d282221a9 · Record review: source checked
- ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Original source · PMC10723403.1
Technical metadata and extraction receipts
Stable ID: lit-017
- areas
- proteins-complexes
- tasks
- Zero-shot substitution mutation effects: stability
- printed value
- 0.488
- numeric value
- 0.488
- metric
- Mean Spearman rho
- metric direction
- unknown
- unit
- unitless
- uncertainty
- Not reported
- source locator
- Table A7, ESM-2 (15B) row, Stability column
- review
- method: independent_ai_table_review; reviewer: Codex omics research agent; independent source-table review, not human review; reviewed at: 2026-09-16T10:41:16.517323+00:00; notes: Table A7 is zero-shot substitution DMS grouped by function. Stability is the fifth numeric column; model-type row spans do not change its placement. This verifies the central score at its source location, not every metadata field or an experimental reproduction.; evidence: {"table_xml_id": "T11", "row_cells": ["ESM-2 (15B)", "0.405", "0.318", "0.425", "0.388", "0.488", "0.405"], "selected_cell_zero_based": 5, "selected_cell_xml": "<td align=\"center\" valign=\"top\" rowspan=\"1\" colspan=\"1\">0.488</td>", "caption": "ProteinGym - Zero-shot substitution DMS benchmark by function typeAverage Spearman’s rank correlation between model scores and experimental measurements on the ProteinGym substitution benchmark, separated into five functional categories (Activity, Binding, Organismal Fitness, Stability and Expression). ‘All’ is the average of all the categories."}; artifact sha256: 4519641f13271bdd09b166e7d93232f22542489bc52a25b5a1628c3df8badce1; retrieval url: https://www.ebi.ac.uk/europepmc/webservices/rest/PMC10723403/fullTextXML
- legacy id
- lit-017
- legacy row
- id: lit-017; paper id: proteingym-2023; domain id: proteins-complexes; task: Zero-shot substitution mutation effects: stability; model: ESM-2; model version: 15B; dataset: ProteinGym substitution DMS: stability assays; dataset version: Not reported; split: Not reported; metric: Mean Spearman rho; value: 0.488; unit: unitless; uncertainty: Not reported; protocol: Zero-shot mutation scores; average Spearman across stability-category assays.; source locator: Table A7, ESM-2 (15B) row, Stability column; source url: https://pmc.ncbi.nlm.nih.gov/articles/PMC10723403/; evaluation origin: independent_paper; reviewed utc: 2026-09-15T23:25:00Z
- missing metadata
- dataset version: not_reported_in_legacy_extract; split: not_reported_in_legacy_extract; uncertainty: not_reported_in_legacy_extract