rewire.it
Dataset subset

ProteinGym v1.3 AMFR substitution assay full assay

Score all 2,972 variants in AMFR_HUMAN_Tsuboyama_2023_4G3O with ESM-2 8M masked marginals. This single-assay result is not a full ProteinGym track or suite score.

Evaluation results

1 evaluation · 5 metric rows. Different protocols are not a single leaderboard.

Filter evaluations

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configurationProtocol and datasetFindingEvidence and details
Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)Protocol: ProteinGym v1.3 AMFR substitution assay
Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay
0.394 AUC
dimensionless · higher

Uncertainty: Not reported

Coverage: 2972/2972

Rewire evaluation · source checked
Methods, coverage and source

ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay

ProteinGym v1.3 AMFR substitution assay

Aggregation: Not reported

ESM-2 8M masked-marginal scoring: local execution report (20 September 2026); ESM-2 8M masked-marginal scoring: automated execution audit · protocol_results.per_assay.AMFR_HUMAN_Tsuboyama_2023_4G3O.metrics.AUC
Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)Protocol: ProteinGym v1.3 AMFR substitution assay
Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay
-0.139 MCC
dimensionless · higher

Uncertainty: Not reported

Coverage: 2972/2972

Rewire evaluation · source checked
Methods, coverage and source

ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay

ProteinGym v1.3 AMFR substitution assay

Aggregation: Not reported

ESM-2 8M masked-marginal scoring: local execution report (20 September 2026); ESM-2 8M masked-marginal scoring: automated execution audit · protocol_results.per_assay.AMFR_HUMAN_Tsuboyama_2023_4G3O.metrics.MCC
Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)Protocol: ProteinGym v1.3 AMFR substitution assay
Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay
0.440 NDCG
dimensionless · higher

Uncertainty: Not reported

Coverage: 2972/2972

Rewire evaluation · source checked
Methods, coverage and source

ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay

ProteinGym v1.3 AMFR substitution assay

Aggregation: Not reported

ESM-2 8M masked-marginal scoring: local execution report (20 September 2026); ESM-2 8M masked-marginal scoring: automated execution audit · protocol_results.per_assay.AMFR_HUMAN_Tsuboyama_2023_4G3O.metrics.NDCG
Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)Protocol: ProteinGym v1.3 AMFR substitution assay
Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay
-0.209 Spearman
dimensionless · higher

Uncertainty: Not reported

Coverage: 2972/2972

Rewire evaluation · source checked
Methods, coverage and source

ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay

ProteinGym v1.3 AMFR substitution assay

Aggregation: Not reported

ESM-2 8M masked-marginal scoring: local execution report (20 September 2026); ESM-2 8M masked-marginal scoring: automated execution audit · protocol_results.per_assay.AMFR_HUMAN_Tsuboyama_2023_4G3O.metrics.Spearman
Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)Protocol: ProteinGym v1.3 AMFR substitution assay
Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay
0.057 Top_recall
dimensionless · higher

Uncertainty: Not reported

Coverage: 2972/2972

Rewire evaluation · source checked
Methods, coverage and source

ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay

ProteinGym v1.3 AMFR substitution assay

Aggregation: Not reported

ESM-2 8M masked-marginal scoring: local execution report (20 September 2026); ESM-2 8M masked-marginal scoring: automated execution audit · protocol_results.per_assay.AMFR_HUMAN_Tsuboyama_2023_4G3O.metrics.Top_recall

Source checking is not independent reproduction. Release 2026-09-22-f58a0f1d267f.

Subset and evaluation context

This record describes a particular subset or cohort used in an evaluation. Its results do not describe the full dataset.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

22 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-22-f58a0f1d267f
Property and statementOriginal source and locationReview and provenance
attributes.dataset_id

proteingym-dms-substitutions-v1.3

Context-only references
ESM-2 8M masked-marginal scoring: local execution report (20 September 2026)

Original source ↗

coverage; protocol_configuration; provenance

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.dataset_id

Source artifact SHA-256: 6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.provenance.assay_sha256.AMFR_HUMAN_Tsuboyama_2023_4G3O

dd911d925eca79329a496eb6cb7b181ab448e15db244daf1c1f9c01e8a704c2a

Context-only references
ESM-2 8M masked-marginal scoring: local execution report (20 September 2026)

Original source ↗

coverage; protocol_configuration; provenance

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.provenance.assay_sha256.AMFR_HUMAN_Tsuboyama_2023_4G3O

Source artifact SHA-256: 6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.provenance.data_verification

local_bytes_hashed_not_independently_source_verified

Context-only references
ESM-2 8M masked-marginal scoring: local execution report (20 September 2026)

Original source ↗

coverage; protocol_configuration; provenance

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.provenance.data_verification

Source artifact SHA-256: 6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.provenance.expected_hashes_checked

["AMFR_HUMAN_Tsuboyama_2023_4G3O"]

Context-only references
ESM-2 8M masked-marginal scoring: local execution report (20 September 2026)

Original source ↗

coverage; protocol_configuration; provenance

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.provenance.expected_hashes_checked

Source artifact SHA-256: 6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.provenance.reference_sha256

a8f498011532a74aa9fe556a50555a75e928c5837d19c06a87592ae04049b308

Context-only references
ESM-2 8M masked-marginal scoring: local execution report (20 September 2026)

Original source ↗

coverage; protocol_configuration; provenance

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.provenance.reference_sha256

Source artifact SHA-256: 6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.provenance.source_url

https://github.com/OATML-Markslab/ProteinGym/tree/144fe22b07dfaeec2b366f2346203a9838a55b4c

Context-only references
ESM-2 8M masked-marginal scoring: local execution report (20 September 2026)

Original source ↗

coverage; protocol_configuration; provenance

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.provenance.source_url

Source artifact SHA-256: 6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.provenance.upstream_revision

144fe22b07dfaeec2b366f2346203a9838a55b4c

Context-only references
ESM-2 8M masked-marginal scoring: local execution report (20 September 2026)

Original source ↗

coverage; protocol_configuration; provenance

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.provenance.upstream_revision

Source artifact SHA-256: 6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.selected_assay

AMFR_HUMAN_Tsuboyama_2023_4G3O

Context-only references
ESM-2 8M masked-marginal scoring: local execution report (20 September 2026)

Original source ↗

coverage; protocol_configuration; provenance

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.selected_assay

Source artifact SHA-256: 6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.source_evidence.archive_sha256

3a83766254ac9ac9984ec25cb73c6e010ea4418f5e35f143933e6b6e6473b921

Context-only references
ESM-2 8M masked-marginal scoring: local execution report (20 September 2026)

Original source ↗

coverage; protocol_configuration; provenance

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.source_evidence.archive_sha256

Source artifact SHA-256: 6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.source_evidence.archive_size_bytes

43021128

Context-only references
ESM-2 8M masked-marginal scoring: local execution report (20 September 2026)

Original source ↗

coverage; protocol_configuration; provenance

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.source_evidence.archive_size_bytes

Source artifact SHA-256: 6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Sources and history

View linked audit checks and correction history

Release 2026-09-22-f58a0f1d267f · Record review: source checked

1 source records and release historyDownload this release
Technical metadata and extraction receipts

Stable ID: rewire-dataset-proteingym-amfr-v13

areas
proteins-complexes
version
ProteinGym-v1.3
dataset id
proteingym-dms-substitutions-v1.3
selected assay
AMFR_HUMAN_Tsuboyama_2023_4G3O
source locator
coverage; protocol_configuration; provenance
test count
2972
split counts
Not reported
provenance
assay sha256: AMFR HUMAN Tsuboyama 2023 4G3O: dd911d925eca79329a496eb6cb7b181ab448e15db244daf1c1f9c01e8a704c2a; data verification: local_bytes_hashed_not_independently_source_verified; expected hashes checked: AMFR_HUMAN_Tsuboyama_2023_4G3O; reference sha256: a8f498011532a74aa9fe556a50555a75e928c5837d19c06a87592ae04049b308; source url: https://github.com/OATML-Markslab/ProteinGym/tree/144fe22b07dfaeec2b366f2346203a9838a55b4c; upstream revision: 144fe22b07dfaeec2b366f2346203a9838a55b4c
source evidence
retrieved at: 2026-09-20T20:45:43.949575+00:00; url: https://marks.hms.harvard.edu/proteingym/ProteinGym_v1.3/DMS_ProteinGym_substitutions.zip; archive sha256: 3a83766254ac9ac9984ec25cb73c6e010ea4418f5e35f143933e6b6e6473b921; archive size bytes: 43021128; member: DMS_ProteinGym_substitutions/AMFR_HUMAN_Tsuboyama_2023_4G3O.csv; member sha256: dd911d925eca79329a496eb6cb7b181ab448e15db244daf1c1f9c01e8a704c2a; hash status: observed_download_bytes_not_independently_published_checksums
subset scope
All selected test rows; this is not the whole benchmark suite.
Related records

Suggest a correction