Model type
Protein sequence transformer; this record is the paper-specific evaluated configuration.
Configuration
ESM-2
This protein model is evaluated as a zero-shot mutation-effect scorer in ProteinGym.
Sources
ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · ProteinGym benchmarks (paragraph 1); Introduction (paragraph 4)How it worksEvaluated procedure (conceptual)
Conceptual input–method–output guide. Check the procedure text and linked evaluation for fitted components, additional inputs and exact settings.
Sources
ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · ProteinGym benchmarks/Dataset types (paragraph 1); Appendix/A Appendix/A.2 Limitations/Deep mutational scans (paragraph 1)At a glance
Biological inputs
Variant protein sequences; inverse-folding methods additionally require structures
Sources
ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Appendix/A Appendix/A.4 Baselines/A.4.1 Zero-shot baselines/Inverse Folding models (paragraph 1); Appendix/A Appendix/A.1 Social Impact (paragraph 2)Outputs
Mutation-effect scores and ranked variants
Sources
ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Appendix/A Appendix/A.4 Baselines/A.4.3 Clinical baselines (paragraph 1); Appendix/A Appendix/A.3 Datasets/A.3.2 Clinical datasets (paragraph 1)limited source coverage · Automated source review, 2026-09-16. All specifications and missing details
Evaluations and results
Release 2026-09-17-d277315f7d76 · 1 evaluation · 1 metric row. Different protocols are not a single leaderboard.
| Metric and finding | Coverage and uncertainty | Evidence |
|---|---|---|
| ESM-2: Zero-shot substitution mutation effects: stability Configuration: ESM-2Task: Zero-shot substitution mutation effects: stabilityDataset: ProteinGym substitution DMS: stability assays Zero-shot mutation scores; average Spearman across stability-category assays. Independent external evaluation · Evaluation metadata: needs review | ||
| 0.488 Mean Spearman rho Unit: unitless · Direction: unknown | Uncertainty: not reported in legacy extract Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Table A7, ESM-2 (15B) row, Stability column Source checking is not independent reproduction. |
How it works
How the evaluated method works
The benchmark converts model sequence or structure-conditioned scores into variant rankings and compares them with experimental deep-mutational-scanning measurements.
Sources
ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · ProteinGym benchmarks/Dataset types (paragraph 1); Appendix/A Appendix/A.2 Limitations/Deep mutational scans (paragraph 1)Underlying method and version boundaries
ESM-2 is a transformer protein language-model family. The official repository exposes residue embeddings, sequence-level pooling and models at several sizes; the study configuration determines which of these is evaluated.
Sources
facebookresearch/esm README.md · README.md; introduction, model description, pretrained-model and usage sections at pinned revisionWhat was evaluated
The linked evaluation record identifies ESM-2: Zero-shot substitution mutation effects: stability. Its dataset, split, adaptation and evidence origin remain attached to the reported results.
Sources
ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · The named evaluation’s methods and comparison table; exact preserved evaluation IDs: evaluation-lit-017Strengths and limitations
Strengths and considerations
- Uses standardised evaluation across diverse experimental assays.
Sources
ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Introduction (paragraph 2); Abstract (paragraph 1)
Limitations and conditions
- Different models receive different evolutionary/structural information; assay fitness, organismal fitness and clinical pathogenicity are not interchangeable endpoints.
Sources
ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · ProteinGym benchmarks (paragraph 1); Appendix/A Appendix/A.3 Datasets/A.3.1 DMS assays/Processing of large thermostability dataset (paragraph 2)
Profile review details
Primary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction.
Stable record: reported-model-d326e3c4e3ba20Specifications
Inputs, training, access and other details
Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Model type | Protein sequence transformer; this record is the paper-specific evaluated configuration.Sourcesfacebookresearch/esm README.md · README.md model description |
| Architecture / procedure | The benchmark converts model sequence or structure-conditioned scores into variant rankings and compares them with experimental deep-mutational-scanning measurements.SourcesProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · ProteinGym benchmarks/Dataset types (paragraph 1); Appendix/A Appendix/A.2 Limitations/Deep mutational scans (paragraph 1) |
| Biological inputs | Variant protein sequences; inverse-folding methods additionally require structuresSourcesProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Appendix/A Appendix/A.4 Baselines/A.4.1 Zero-shot baselines/Inverse Folding models (paragraph 1); Appendix/A Appendix/A.1 Social Impact (paragraph 2) |
| Outputs | Mutation-effect scores and ranked variantsSourcesProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Appendix/A Appendix/A.4 Baselines/A.4.3 Clinical baselines (paragraph 1); Appendix/A Appendix/A.3 Datasets/A.3.2 Clinical datasets (paragraph 1) |
| Parameters | 15 billion parameters, as identified for this rowSourcesProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Appendix/A Appendix/A.5 Detailed performance results/A.5.4 Clinical indel benchmarks (paragraph 1); Appendix/A Appendix/A.5 Detailed performance results/A.5.3 Clinical substitution benchmarks (paragraph 1) |
| Known versions / configuration | 15BSourcesProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Appendix/A Appendix/A.5 Detailed performance results/A.5.4 Clinical indel benchmarks (paragraph 1); Appendix/A Appendix/A.5 Detailed performance results/A.5.3 Clinical substitution benchmarks (paragraph 1) |
| Training data / fitting | The linked ProteinGym row is a zero-shot evaluation of a pretrained protein language model: no DMS phenotype labels are used to fit this score. Protein-family-specific alignment training and supervised assay predictors are separate benchmark settings.SourcesProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Evaluation framework / Zero-shot benchmarks / Baselines; Appendix A.4.1 |
| Context limits | A maximum input/context length for this exact evaluated configuration is not established by the inspected sources. · Not reported in inspected sourcesSources (2)ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction; facebookresearch/esm README.md · ProteinGym benchmarks/Model training regime; inspected for explicit maximum input length (dataset lengths and family-wide limits are not substituted); README.md at pinned repository revision |
| Access | Official upstream implementation and usage documentation: https://github.com/facebookresearch/esm/blob/2b369911bb5b4b0dda914521b9475cad1656b2ac/README.md. This pinned documentation revision is not automatically the evaluated weight revision.Sourcesfacebookresearch/esm README.md · README.md; installation, model download and usage instructions |
| Code licence | MIT (upstream repository code at the cited revision; this does not establish every dependency or historical checkpoint licence).Sourcesfacebookresearch/esm LICENSE · LICENSE; complete licence text |
| Weights licence | The inspected model-access documentation does not explicitly identify terms for this exact evaluated checkpoint or fitted head; repository code terms are shown separately. · Not reported in inspected sourcesSourcesfacebookresearch/esm README.md · README.md; checkpoint/access documentation and licence scope |
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
21 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Diagram caption Conceptual input–method–output guide. Check the procedure text and linked evaluation for fitted components, additional inputs and exact settings. Individual claims | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction ProteinGym benchmarks/Dataset types (paragraph 1); Appendix/A Appendix/A.2 Limitations/Deep mutational scans (paragraph 1) Version: PMC10723403.1 | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram steps ["Variant protein sequences","ESM-2","Mutation-effect scores and ranked variants"] Individual claims | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction ProteinGym benchmarks/Dataset types (paragraph 1); Appendix/A Appendix/A.2 Limitations/Deep mutational scans (paragraph 1) Version: PMC10723403.1 | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Evaluated procedure (conceptual) Individual claims | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction ProteinGym benchmarks/Dataset types (paragraph 1); Appendix/A Appendix/A.2 Limitations/Deep mutational scans (paragraph 1) Version: PMC10723403.1 | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Model type Protein sequence transformer; this record is the paper-specific evaluated configuration. Individual claims | facebookresearch/esm README.md README.md model description Version: 2b369911bb5b4b0dda914521b9475cad1656b2ac | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Architecture / procedure The benchmark converts model sequence or structure-conditioned scores into variant rankings and compares them with experimental deep-mutational-scanning measurements. Individual claims | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction ProteinGym benchmarks/Dataset types (paragraph 1); Appendix/A Appendix/A.2 Limitations/Deep mutational scans (paragraph 1) Version: PMC10723403.1 | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Weights licence The inspected model-access documentation does not explicitly identify terms for this exact evaluated checkpoint or fitted head; repository code terms are shown separately. Individual claims | facebookresearch/esm README.md README.md; checkpoint/access documentation and licence scope Version: 2b369911bb5b4b0dda914521b9475cad1656b2ac | unreported automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Biological inputs Variant protein sequences; inverse-folding methods additionally require structures Individual claims | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction Appendix/A Appendix/A.4 Baselines/A.4.1 Zero-shot baselines/Inverse Folding models (paragraph 1); Appendix/A Appendix/A.1 Social Impact (paragraph 2) Version: PMC10723403.1 | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Outputs Mutation-effect scores and ranked variants Individual claims | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction Appendix/A Appendix/A.4 Baselines/A.4.3 Clinical baselines (paragraph 1); Appendix/A Appendix/A.3 Datasets/A.3.2 Clinical datasets (paragraph 1) Version: PMC10723403.1 | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Parameters 15 billion parameters, as identified for this row Individual claims | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction Appendix/A Appendix/A.5 Detailed performance results/A.5.4 Clinical indel benchmarks (paragraph 1); Appendix/A Appendix/A.5 Detailed performance results/A.5.3 Clinical substitution benchmarks (paragraph 1) Version: PMC10723403.1 | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Known versions / configuration 15B Individual claims | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction Appendix/A Appendix/A.5 Detailed performance results/A.5.4 Clinical indel benchmarks (paragraph 1); Appendix/A Appendix/A.5 Detailed performance results/A.5.3 Clinical substitution benchmarks (paragraph 1) Version: PMC10723403.1 | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-09-17-d277315f7d76 · Record review: needs review
3 source records and release history
- ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction · Original source · PMC10723403.1
- facebookresearch/esm README.md · Original source · 2b369911bb5b4b0dda914521b9475cad1656b2ac
- facebookresearch/esm LICENSE · Original source · 2b369911bb5b4b0dda914521b9475cad1656b2ac
Technical metadata and extraction receipts
Stable ID: reported-model-d326e3c4e3ba20
- areas
- proteins-complexes
- entity level
- method
- version
- 15B
- reported name
- ESM-2
- historical missing metadata
- checkpoint revision: not_reported_in_legacy_extract; training data: not_reported_in_legacy_extract; licence: not_reported_in_legacy_extract
- metadata review scope
- historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
- legacy kinds
- model
- entity classification
- review date: 2026-09-17; rationale: This source-scoped entry preserves the method/configuration actually named in an evaluation. It is neither a global family identity nor proof of an immutable checkpoint; the linked evaluation retains adaptation, fitting and scoring details.; source ids: proteingym-2023; evidence-reported-base-esm-readme-md; source locator: ProteinGym benchmarks/Dataset types (paragraph 1); Appendix/A Appendix/A.2 Limitations/Deep mutational scans (paragraph 1) | README.md model description | ProteinGym benchmarks (paragraph 1); Introduction (paragraph 4); ambiguities: Configuration means the source-labelled evaluated identity. It does not establish missing checkpoint hashes, default settings or equivalence to same-named records in other papers.