Model type
Protein sequence transformer; this record is the paper-specific evaluated configuration.
Configuration
ESM-2
ESM-2 is the sequence-only baseline in the Protein Structure Transformer study.
Sources
Endowing protein language models with structural knowledge · 1 Introduction (paragraph 5); 2 Related work (paragraph 1)How it worksEvaluated procedure (conceptual)
Conceptual input–method–output guide. Check the procedure text and linked evaluation for fitted components, additional inputs and exact settings.
Sources
Endowing protein language models with structural knowledge · 4 Experiments/4.1 Experimental setup/4.1.5 Task-specific models (paragraph 1); 2 Related work/2.1 Sequence-based models (paragraph 1)At a glance
Biological inputs
Protein amino-acid sequences
Sources
Endowing protein language models with structural knowledge · 3 Methods/3.1 Evolutionary scale modeling (paragraph 3); 2 Related work (paragraph 1)Outputs
Protein embeddings and task-specific predictions
Sources
Endowing protein language models with structural knowledge · 4 Experiments/4.1 Experimental setup/4.1.5 Task-specific models (paragraph 1); 1 Introduction (paragraph 1)limited source coverage · Automated source review, 2026-09-16. All specifications and missing details
Evaluations and results
Release 2026-09-17-d277315f7d76 · 1 evaluation · 1 metric row. Different protocols are not a single leaderboard.
| Metric and finding | Coverage and uncertainty | Evidence |
|---|---|---|
| ESM-2: Zero-shot variant effect prediction Zero-shot VEP; paper averages absolute Spearman correlations. Independent external evaluation · Evaluation metadata: needs review | ||
| 0.489 Mean |Spearman rho| Unit: unitless · Direction: unknown | Uncertainty: not reported in legacy extract Scored: Not reported · Eligible: Not reported | source checkedEndowing protein language models with structural knowledge · Table 2, ESM-2 row, Zero-shot VEP Mean |ρ| column Source checking is not independent reproduction. |
How it works
How the evaluated method works
A pretrained protein transformer produces sequence representations without the structural adapters added by PST.
Sources
Endowing protein language models with structural knowledge · 4 Experiments/4.1 Experimental setup/4.1.5 Task-specific models (paragraph 1); 2 Related work/2.1 Sequence-based models (paragraph 1)Underlying method and version boundaries
ESM-2 is a transformer protein language-model family. The official repository exposes residue embeddings, sequence-level pooling and models at several sizes; the study configuration determines which of these is evaluated.
Sources
facebookresearch/esm README.md · README.md; introduction, model description, pretrained-model and usage sections at pinned revisionWhat was evaluated
The linked evaluation record identifies ESM-2: Zero-shot variant effect prediction. Its dataset, split, adaptation and evidence origin remain attached to the reported results.
Sources
Endowing protein language models with structural knowledge · The named evaluation’s methods and comparison table; exact preserved evaluation IDs: evaluation-lit-024Strengths and limitations
Strengths and considerations
- Controls for the value of the pretrained sequence representation when testing added structural information.
Sources
Endowing protein language models with structural knowledge · 1 Introduction (paragraph 7); 4 Experiments/4.4 Ablation studies (paragraph 1)
Limitations and conditions
- PST variants use additional structure data and adapters; their scores and parameter overhead must not be attributed to this unmodified baseline.
Sources
Endowing protein language models with structural knowledge · 4 Experiments/4.4 Ablation studies/4.4.3 Pretraining only structure extractors almost matches full-model pretraining performance (paragraph 2); 4 Experiments/4.4 Ablation studies/4.4.4 PST gains are due to structural information (paragraph 1)
Profile review details
Primary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction.
Stable record: reported-model-d25dab1a9c4fffSpecifications
Inputs, training, access and other details
Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Model type | Protein sequence transformer; this record is the paper-specific evaluated configuration.Sourcesfacebookresearch/esm README.md · README.md model description |
| Architecture / procedure | A pretrained protein transformer produces sequence representations without the structural adapters added by PST.SourcesEndowing protein language models with structural knowledge · 4 Experiments/4.1 Experimental setup/4.1.5 Task-specific models (paragraph 1); 2 Related work/2.1 Sequence-based models (paragraph 1) |
| Biological inputs | Protein amino-acid sequencesSourcesEndowing protein language models with structural knowledge · 3 Methods/3.1 Evolutionary scale modeling (paragraph 3); 2 Related work (paragraph 1) |
| Outputs | Protein embeddings and task-specific predictionsSourcesEndowing protein language models with structural knowledge · 4 Experiments/4.1 Experimental setup/4.1.5 Task-specific models (paragraph 1); 1 Introduction (paragraph 1) |
| Parameters | An aggregate parameter total for this exact evaluated configuration is not established by the inspected sources. · Not reported in inspected sourcesSources (2)Endowing protein language models with structural knowledge; facebookresearch/esm README.md · 3 Methods; 3 Methods/3.1 Evolutionary scale modeling; 3 Methods/3.1 Evolutionary scale modeling/3.1.1 ESM-2 model architecture; 3 Methods/3.2 Protein structure transformer; 3 Methods/3.2 Protein structure transformer/3.2.1 Protein graph representation; 3 Methods/3.2 Protein structure transformer/3.2.2 Protein structure transformer construction; 3 Methods/3.2 Protein structure transformer/3.2.3 Pretraining the PST; 4 Experiments/4.1 Experimental setup/4.1.4 Pretraining; inspected for aggregate parameter count (component sizes are not added without an exact configuration); README.md at pinned repository revision |
| Known versions / configuration | ESM-2 is the comparison-table label; that label does not specify an immutable weight revision. · Not reported in inspected sourcesSourcesEndowing protein language models with structural knowledge · Model identification in the comparison table and corresponding Methods; immutable checkpoint revision is not supplied by the table label. |
| Training data / fitting | Masked-language-model pretraining samples approximately 43 million UniRef50 clusters from 138 million UniRef90 sequences, exposing ESM-2 to about 65 million distinct sequences. The sequence-only comparator remains distinct from PST’s structure-adaptation stage.SourcesEndowing protein language models with structural knowledge · Methods / Evolutionary scale modeling |
| Context limits | A maximum input/context length for this exact evaluated configuration is not established by the inspected sources. · Not reported in inspected sourcesSources (2)Endowing protein language models with structural knowledge; facebookresearch/esm README.md · 3 Methods; 3 Methods/3.1 Evolutionary scale modeling; 3 Methods/3.1 Evolutionary scale modeling/3.1.1 ESM-2 model architecture; 3 Methods/3.2 Protein structure transformer; 3 Methods/3.2 Protein structure transformer/3.2.1 Protein graph representation; 3 Methods/3.2 Protein structure transformer/3.2.2 Protein structure transformer construction; 3 Methods/3.2 Protein structure transformer/3.2.3 Pretraining the PST; 4 Experiments/4.1 Experimental setup/4.1.4 Pretraining; inspected for explicit maximum input length (dataset lengths and family-wide limits are not substituted); README.md at pinned repository revision |
| Access | Official upstream implementation and usage documentation: https://github.com/facebookresearch/esm/blob/2b369911bb5b4b0dda914521b9475cad1656b2ac/README.md. This pinned documentation revision is not automatically the evaluated weight revision.Sourcesfacebookresearch/esm README.md · README.md; installation, model download and usage instructions |
| Code licence | MIT (upstream repository code at the cited revision; this does not establish every dependency or historical checkpoint licence).Sourcesfacebookresearch/esm LICENSE · LICENSE; complete licence text |
| Weights licence | The inspected model-access documentation does not explicitly identify terms for this exact evaluated checkpoint or fitted head; repository code terms are shown separately. · Not reported in inspected sourcesSourcesfacebookresearch/esm README.md · README.md; checkpoint/access documentation and licence scope |
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
22 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Diagram caption Conceptual input–method–output guide. Check the procedure text and linked evaluation for fitted components, additional inputs and exact settings. Individual claims | Endowing protein language models with structural knowledge 4 Experiments/4.1 Experimental setup/4.1.5 Task-specific models (paragraph 1); 2 Related work/2.1 Sequence-based models (paragraph 1) Version: PMC12603367.1 | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram steps ["Protein amino-acid sequences","ESM-2","Protein embeddings and task-specific predictions"] Individual claims | Endowing protein language models with structural knowledge 4 Experiments/4.1 Experimental setup/4.1.5 Task-specific models (paragraph 1); 2 Related work/2.1 Sequence-based models (paragraph 1) Version: PMC12603367.1 | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Evaluated procedure (conceptual) Individual claims | Endowing protein language models with structural knowledge 4 Experiments/4.1 Experimental setup/4.1.5 Task-specific models (paragraph 1); 2 Related work/2.1 Sequence-based models (paragraph 1) Version: PMC12603367.1 | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Model type Protein sequence transformer; this record is the paper-specific evaluated configuration. Individual claims | facebookresearch/esm README.md README.md model description Version: 2b369911bb5b4b0dda914521b9475cad1656b2ac | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Architecture / procedure A pretrained protein transformer produces sequence representations without the structural adapters added by PST. Individual claims | Endowing protein language models with structural knowledge 4 Experiments/4.1 Experimental setup/4.1.5 Task-specific models (paragraph 1); 2 Related work/2.1 Sequence-based models (paragraph 1) Version: PMC12603367.1 | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Weights licence The inspected model-access documentation does not explicitly identify terms for this exact evaluated checkpoint or fitted head; repository code terms are shown separately. Individual claims | facebookresearch/esm README.md README.md; checkpoint/access documentation and licence scope Version: 2b369911bb5b4b0dda914521b9475cad1656b2ac | unreported automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Biological inputs Protein amino-acid sequences Individual claims | Endowing protein language models with structural knowledge 3 Methods/3.1 Evolutionary scale modeling (paragraph 3); 2 Related work (paragraph 1) Version: PMC12603367.1 | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Outputs Protein embeddings and task-specific predictions Individual claims | Endowing protein language models with structural knowledge 4 Experiments/4.1 Experimental setup/4.1.5 Task-specific models (paragraph 1); 1 Introduction (paragraph 1) Version: PMC12603367.1 | source checked automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Parameters An aggregate parameter total for this exact evaluated configuration is not established by the inspected sources. Individual claims | facebookresearch/esm README.md 3 Methods; 3 Methods/3.1 Evolutionary scale modeling; 3 Methods/3.1 Evolutionary scale modeling/3.1.1 ESM-2 model architecture; 3 Methods/3.2 Protein structure transformer; 3 Methods/3.2 Protein structure transformer/3.2.1 Protein graph representation; 3 Methods/3.2 Protein structure transformer/3.2.2 Protein structure transformer construction; 3 Methods/3.2 Protein structure transformer/3.2.3 Pretraining the PST; 4 Experiments/4.1 Experimental setup/4.1.4 Pretraining; inspected for aggregate parameter count (component sizes are not added without an exact configuration); README.md at pinned repository revision Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 2b369911bb5b4b0dda914521b9475cad1656b2ac | unreported automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Parameters An aggregate parameter total for this exact evaluated configuration is not established by the inspected sources. Individual claims | Endowing protein language models with structural knowledge 3 Methods; 3 Methods/3.1 Evolutionary scale modeling; 3 Methods/3.1 Evolutionary scale modeling/3.1.1 ESM-2 model architecture; 3 Methods/3.2 Protein structure transformer; 3 Methods/3.2 Protein structure transformer/3.2.1 Protein graph representation; 3 Methods/3.2 Protein structure transformer/3.2.2 Protein structure transformer construction; 3 Methods/3.2 Protein structure transformer/3.2.3 Pretraining the PST; 4 Experiments/4.1 Experimental setup/4.1.4 Pretraining; inspected for aggregate parameter count (component sizes are not added without an exact configuration); README.md at pinned repository revision Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: PMC12603367.1 | unreported automated source review · 2026-09-16 Audit detailsPrimary full text and the available official implementation/model documentation were inspected. Explanatory claims are source-backed; unresolved exact-configuration metadata is labelled explicitly. This is automated review, not a human review or independent benchmark reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-09-17-d277315f7d76 · Record review: needs review
3 source records and release history
- Endowing protein language models with structural knowledge · Original source · PMC12603367.1
- facebookresearch/esm README.md · Original source · 2b369911bb5b4b0dda914521b9475cad1656b2ac
- facebookresearch/esm LICENSE · Original source · 2b369911bb5b4b0dda914521b9475cad1656b2ac
Technical metadata and extraction receipts
Stable ID: reported-model-d25dab1a9c4fff
- areas
- proteins-complexes
- entity level
- method
- version
- Not reported
- reported name
- ESM-2
- historical missing metadata
- version: not_reported_in_legacy_extract; checkpoint revision: not_reported_in_legacy_extract; training data: not_reported_in_legacy_extract; licence: not_reported_in_legacy_extract
- metadata review scope
- historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
- legacy kinds
- model
- entity classification
- review date: 2026-09-17; rationale: This source-scoped entry preserves the method/configuration actually named in an evaluation. It is neither a global family identity nor proof of an immutable checkpoint; the linked evaluation retains adaptation, fitting and scoring details.; source ids: pst-2025; evidence-reported-base-esm-readme-md; source locator: 4 Experiments/4.1 Experimental setup/4.1.5 Task-specific models (paragraph 1); 2 Related work/2.1 Sequence-based models (paragraph 1) | README.md model description | 1 Introduction (paragraph 5); 2 Related work (paragraph 1); ambiguities: Configuration means the source-labelled evaluated identity. It does not establish missing checkpoint hashes, default settings or equivalence to same-named records in other papers.