ProteinGym (PFMBench split)
The split of ProteinGym that PFMBench evaluated on. The upstream dataset release is not catalogued here, so no claim is made that this matches its original splits.
Subset and evaluation context
This record describes a particular subset or cohort used in an evaluation. Its results do not describe the full dataset.
Evaluation results
Release 2026-09-17-134cd1815de8 · 9 evaluations · 9 metric rows. Different protocols are not a single leaderboard. Where several source tables report the same metric, the published comparisons above offer a pooled view that names what it does not hold constant.
| Metric and finding | Coverage and uncertainty | Evidence |
|---|---|---|
| DPLM on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction Configuration: DPLMTask: PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect predictionDataset subset: ProteinGym (PFMBench split) Scored zero-shot, with no fine-tuning, as reported in PFMBench Table 4. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.42922 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedpfmbench primary benchmark evidence · Table 4, row(DPLM [ 68 ]), column(ProteinGym) Source checking is not independent reproduction. |
| ESM-2 on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction Configuration: ESM-2Task: PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect predictionDataset subset: ProteinGym (PFMBench split) Scored zero-shot, with no fine-tuning, as reported in PFMBench Table 4. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.43904 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedpfmbench primary benchmark evidence · Table 4, row(ESM-2 [ 35 ]), column(ProteinGym) Source checking is not independent reproduction. |
| ESM-C on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction Configuration: ESM-CTask: PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect predictionDataset subset: ProteinGym (PFMBench split) Scored zero-shot, with no fine-tuning, as reported in PFMBench Table 4. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.43422 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedpfmbench primary benchmark evidence · Table 4, row(ESM-C), column(ProteinGym) Source checking is not independent reproduction. |
| ESM3 on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction Configuration: ESM3Task: PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect predictionDataset subset: ProteinGym (PFMBench split) Scored zero-shot, with no fine-tuning, as reported in PFMBench Table 4. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.41401 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedpfmbench primary benchmark evidence · Table 4, row(ESM3 [ 18 ]), column(ProteinGym) Source checking is not independent reproduction. |
| PGLM on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction Configuration: PGLMTask: PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect predictionDataset subset: ProteinGym (PFMBench split) Scored zero-shot, with no fine-tuning, as reported in PFMBench Table 4. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.39750 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedpfmbench primary benchmark evidence · Table 4, row(PGLM [ 5 ]), column(ProteinGym) Source checking is not independent reproduction. |
| ProtGPT2 on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction Configuration: ProtGPT2Task: PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect predictionDataset subset: ProteinGym (PFMBench split) Scored zero-shot, with no fine-tuning, as reported in PFMBench Table 4. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.18962 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedpfmbench primary benchmark evidence · Table 4, row(ProtGPT2 [ 12 ]), column(ProteinGym) Source checking is not independent reproduction. |
| ProTrek on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction Configuration: ProTrekTask: PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect predictionDataset subset: ProteinGym (PFMBench split) Scored zero-shot, with no fine-tuning, as reported in PFMBench Table 4. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.35919 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedpfmbench primary benchmark evidence · Table 4, row(ProTrek [ 56 ]), column(ProteinGym) Source checking is not independent reproduction. |
| SaProt on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction Configuration: SaProtTask: PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect predictionDataset subset: ProteinGym (PFMBench split) Scored zero-shot, with no fine-tuning, as reported in PFMBench Table 4. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.45094 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedpfmbench primary benchmark evidence · Table 4, row(SaProt [ 55 ]), column(ProteinGym) Source checking is not independent reproduction. |
| VenusPLM on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction Configuration: VenusPLMTask: PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect predictionDataset subset: ProteinGym (PFMBench split) Scored zero-shot, with no fine-tuning, as reported in PFMBench Table 4. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.43952 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedpfmbench primary benchmark evidence · Table 4, row(VenusPLM [ 59 ]), column(ProteinGym) Source checking is not independent reproduction. |
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
2 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| description The split of ProteinGym that PFMBench evaluated on. The upstream dataset release is not catalogued here, so no claim is made that this matches its original splits. Context-only references | pfmbench primary benchmark evidence No field-specific location recorded Version: 2506.14796v1 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| name ProteinGym (PFMBench split) Context-only references | pfmbench primary benchmark evidence No field-specific location recorded Version: 2506.14796v1 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
Sources and history
Release 2026-09-17-134cd1815de8 · Record review: source checked
1 source records and release history
- pfmbench primary benchmark evidence · Original source · 2506.14796v1
Technical metadata and extraction receipts
Stable ID: pfmbench-dataset-proteingym
- areas
- proteins-complexes
- missing metadata
- version: unreported; url: unextracted
Related records
- dataset: DPLM on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction
- dataset: ESM-2 on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction
- dataset: ESM-C on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction
- dataset: ESM3 on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction
- dataset: PGLM on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction
- dataset: ProtGPT2 on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction
- dataset: ProTrek on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction
- dataset: SaProt on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction
- dataset: VenusPLM on PFMBench PROTEINGYM-ZS: ProteinGym zero-shot variant effect prediction