rewire.it
benchmark · suite

ProteinGym

ProteinGym evaluates mutation-effect predictors against experimental protein assays.

0 evaluations · 0 metric rows

At a glance

Explanatory profile: source reviewed · Automated source review, 2026-09-16. This does not change the review status of its results.

Data, procedure and scoring
PropertyDescription and evidence
Record typeAssay and evaluation suiteOATML-Markslab/ProteinGym official source · README: Overview; Results; Resources; Usage and reproducibility
InputsProtein variants, assay measurements and permitted model inputsOATML-Markslab/ProteinGym official source · README: Overview; Results; Resources; Usage and reproducibility
Outputs and assessmentSpearman and other track-specific metricsOATML-Markslab/ProteinGym official source · README: Overview; Results; Resources; Usage and reproducibility
DatasetsNot extracted or verified for this record.
OrganismsNot extracted or verified for this record.
AssaysNot extracted or verified for this record.
SplitsNot extracted or verified for this record.
AdaptationNot extracted or verified for this record.
BaselinesNot extracted or verified for this record.

How it works

Procedure overview

Conceptual overview of the cited procedure; consult the pinned source for executable settings.

Procedure overviewPin assay release. Then: Score protein variants. Then: Compute assay metrics. Then: Aggregate by protocolPin assay releaseScore protein variantsCompute assay metricsAggregate by protocol
Read the diagram as text
  1. Pin assay release
  2. Score protein variants
  3. Compute assay metrics
  4. Aggregate by protocol
OATML-Markslab/ProteinGym official source · README: Overview; Results; Resources; Usage and reproducibility

Procedure

Score each variant in a pinned assay release, evaluate within assays and then apply the published protein and function-category aggregation. Keep substitutions, indels, zero-shot and supervised tracks separate.

OATML-Markslab/ProteinGym official source · README: Overview; Results; Resources; Usage and reproducibility

Connected tasks and protocols

These associations do not imply identical protocols or interchangeable scores.

Tested models and results

Release 2026-09-16-d74d282221a9 · 0 evaluations · 0 metric rows. Different protocols are not a single leaderboard.

No evaluations linked in this release.

Strengths and limitations

Strengths supported by sources

Limitations and conditions

  • DMS fitness is assay-specific; MSA- and structure-informed methods use different inputs from sequence-only methods.OATML-Markslab/ProteinGym official source · README: Overview; Results; Resources; Usage and reproducibility
Profile review details

Primary-source description checked by an automated research assistant. This is a profile review, not an independent execution or numerical reproduction.

Stable record: discovery-benchmark-proteingym

Applicable tests and references

Applicability is distinct from a completed evaluation.

  • ESM-1v · Proposed association
  • ESM-2 · Proposed association

Sources and history

Release 2026-09-16-d74d282221a9 · Record review: discovered

Download this release
Technical metadata and extraction receipts

Stable ID: discovery-benchmark-proteingym

areas
protein-function
entity level
suite
missing metadata
dataset release: unextracted; metric implementation: unextracted; split manifest: unextracted; version: unextracted
scope note
Specialist molecular or omics evaluation; protocol details require review before numerical comparison.
task
Protein variant effect prediction
version
Not reported
Related records

Suggest a correction