ProteinGym substitution DMS assays (ProteinGym split)
The split of ProteinGym substitution DMS assays that ProteinGym evaluated on. The upstream dataset release is not catalogued here, so no claim is made that this matches its original splits.
Subset and evaluation context
This record describes a particular subset or cohort used in an evaluation. Its results do not describe the full dataset.
Evaluation results
Release 2026-09-17-134cd1815de8 · 185 evaluations · 185 metric rows. Different protocols are not a single leaderboard. Where several source tables report the same metric, the published comparisons above offer a pooled view that names what it does not hold constant.
| Metric and finding | Coverage and uncertainty | Evidence |
|---|---|---|
| CARP (640M) on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC Configuration: CARP (640M)Task: ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUCDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.701 auc Unit: fraction · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(language CARP (640M)), column(Zero-shot substitutions, AUC) Source checking is not independent reproduction. |
| CARP (640M) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC Configuration: CARP (640M)Task: ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCCDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.285 mcc Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(language CARP (640M)), column(Zero-shot substitutions, MCC) Source checking is not independent reproduction. |
| CARP (640M) on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10% Configuration: CARP (640M)Task: ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%Dataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.748 ndcg Unit: fraction · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(language CARP (640M)), column(Zero-shot substitutions, NDCG@10%) Source checking is not independent reproduction. |
| CARP (640M) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: CARP (640M)Task: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.208 recall Unit: fraction · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(language CARP (640M)), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction. |
| CARP (640M) on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman Configuration: CARP (640M)Task: ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, SpearmanDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.368 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(language CARP (640M)), column(Zero-shot substitutions, Spearman) Source checking is not independent reproduction. |
| DeepSequence (ensemble) on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC Configuration: DeepSequence (ensemble)Task: ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUCDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.729 auc Unit: fraction · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(DeepSequence (ensemble)), column(Zero-shot substitutions, AUC) Source checking is not independent reproduction. |
| DeepSequence (ensemble) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC Configuration: DeepSequence (ensemble)Task: ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCCDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.328 mcc Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(DeepSequence (ensemble)), column(Zero-shot substitutions, MCC) Source checking is not independent reproduction. |
| DeepSequence (ensemble) on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10% Configuration: DeepSequence (ensemble)Task: ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%Dataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.776 ndcg Unit: fraction · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(DeepSequence (ensemble)), column(Zero-shot substitutions, NDCG@10%) Source checking is not independent reproduction. |
| DeepSequence (ensemble) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Configuration: DeepSequence (ensemble)Task: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recallDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.226 recall Unit: fraction · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(DeepSequence (ensemble)), column(Zero-shot substitutions, top 10% recall) Source checking is not independent reproduction. |
| DeepSequence (ensemble) on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman Configuration: DeepSequence (ensemble)Task: ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, SpearmanDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.419 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(DeepSequence (ensemble)), column(Zero-shot substitutions, Spearman) Source checking is not independent reproduction. |
| Embed. ESM-1v on ProteinGym SUP-SUB-MSE-AVG: Supervised substitutions, average over splits, MSE Configuration: Embed. ESM-1vTask: ProteinGym SUP-SUB-MSE-AVG: Supervised substitutions, average over splits, MSEDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.787 mse Unit: error · Direction: lower | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, row(Embed. ESM-1v), column(Supervised substitutions, average over splits, MSE) Source checking is not independent reproduction. |
| Embed. ESM-1v on ProteinGym SUP-SUB-MSE-CONTIG: Supervised substitutions, contiguous split, MSE Configuration: Embed. ESM-1vTask: ProteinGym SUP-SUB-MSE-CONTIG: Supervised substitutions, contiguous split, MSEDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.937 mse Unit: error · Direction: lower | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, row(Embed. ESM-1v), column(Supervised substitutions, contiguous split, MSE) Source checking is not independent reproduction. |
| Embed. ESM-1v on ProteinGym SUP-SUB-MSE-MOD: Supervised substitutions, modulo split, MSE Configuration: Embed. ESM-1vTask: ProteinGym SUP-SUB-MSE-MOD: Supervised substitutions, modulo split, MSEDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.861 mse Unit: error · Direction: lower | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, row(Embed. ESM-1v), column(Supervised substitutions, modulo split, MSE) Source checking is not independent reproduction. |
| Embed. ESM-1v on ProteinGym SUP-SUB-MSE-RAND: Supervised substitutions, random split, MSE Configuration: Embed. ESM-1vTask: ProteinGym SUP-SUB-MSE-RAND: Supervised substitutions, random split, MSEDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.563 mse Unit: error · Direction: lower | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, row(Embed. ESM-1v), column(Supervised substitutions, random split, MSE) Source checking is not independent reproduction. |
| Embed. ESM-1v on ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman Configuration: Embed. ESM-1vTask: ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, SpearmanDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.542 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, row(Embed. ESM-1v), column(Supervised substitutions, average over splits, Spearman) Source checking is not independent reproduction. |
| Embed. ESM-1v on ProteinGym SUP-SUB-SPEARMAN-CONTIG: Supervised substitutions, contiguous split, Spearman Configuration: Embed. ESM-1vTask: ProteinGym SUP-SUB-SPEARMAN-CONTIG: Supervised substitutions, contiguous split, SpearmanDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.481 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, row(Embed. ESM-1v), column(Supervised substitutions, contiguous split, Spearman) Source checking is not independent reproduction. |
| Embed. ESM-1v on ProteinGym SUP-SUB-SPEARMAN-MOD: Supervised substitutions, modulo split, Spearman Configuration: Embed. ESM-1vTask: ProteinGym SUP-SUB-SPEARMAN-MOD: Supervised substitutions, modulo split, SpearmanDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.506 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, row(Embed. ESM-1v), column(Supervised substitutions, modulo split, Spearman) Source checking is not independent reproduction. |
| Embed. ESM-1v on ProteinGym SUP-SUB-SPEARMAN-RAND: Supervised substitutions, random split, Spearman Configuration: Embed. ESM-1vTask: ProteinGym SUP-SUB-SPEARMAN-RAND: Supervised substitutions, random split, SpearmanDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.639 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, row(Embed. ESM-1v), column(Supervised substitutions, random split, Spearman) Source checking is not independent reproduction. |
| Embed. MSAT on ProteinGym SUP-SUB-MSE-AVG: Supervised substitutions, average over splits, MSE Configuration: Embed. MSATTask: ProteinGym SUP-SUB-MSE-AVG: Supervised substitutions, average over splits, MSEDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.735 mse Unit: error · Direction: lower | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, row(MSAT), column(Supervised substitutions, average over splits, MSE) Source checking is not independent reproduction. |
| Embed. MSAT on ProteinGym SUP-SUB-MSE-CONTIG: Supervised substitutions, contiguous split, MSE Configuration: Embed. MSATTask: ProteinGym SUP-SUB-MSE-CONTIG: Supervised substitutions, contiguous split, MSEDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.836 mse Unit: error · Direction: lower | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, row(MSAT), column(Supervised substitutions, contiguous split, MSE) Source checking is not independent reproduction. |
| Embed. MSAT on ProteinGym SUP-SUB-MSE-MOD: Supervised substitutions, modulo split, MSE Configuration: Embed. MSATTask: ProteinGym SUP-SUB-MSE-MOD: Supervised substitutions, modulo split, MSEDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.795 mse Unit: error · Direction: lower | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, row(MSAT), column(Supervised substitutions, modulo split, MSE) Source checking is not independent reproduction. |
| Embed. MSAT on ProteinGym SUP-SUB-MSE-RAND: Supervised substitutions, random split, MSE Configuration: Embed. MSATTask: ProteinGym SUP-SUB-MSE-RAND: Supervised substitutions, random split, MSEDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.573 mse Unit: error · Direction: lower | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, row(MSAT), column(Supervised substitutions, random split, MSE) Source checking is not independent reproduction. |
| Embed. MSAT on ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman Configuration: Embed. MSATTask: ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, SpearmanDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.568 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, row(MSAT), column(Supervised substitutions, average over splits, Spearman) Source checking is not independent reproduction. |
| Embed. MSAT on ProteinGym SUP-SUB-SPEARMAN-CONTIG: Supervised substitutions, contiguous split, Spearman Configuration: Embed. MSATTask: ProteinGym SUP-SUB-SPEARMAN-CONTIG: Supervised substitutions, contiguous split, SpearmanDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.525 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, row(MSAT), column(Supervised substitutions, contiguous split, Spearman) Source checking is not independent reproduction. |
| Embed. MSAT on ProteinGym SUP-SUB-SPEARMAN-MOD: Supervised substitutions, modulo split, Spearman Configuration: Embed. MSATTask: ProteinGym SUP-SUB-SPEARMAN-MOD: Supervised substitutions, modulo split, SpearmanDataset subset: ProteinGym substitution DMS assays (ProteinGym split) Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits. Author-reported evaluation · Evaluation metadata: source checked | ||
| 0.538 spearman Unit: correlation · Direction: higher | Uncertainty: Not reported Scored: Not reported · Eligible: Not reported | source checkedProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, row(MSAT), column(Supervised substitutions, modulo split, Spearman) Source checking is not independent reproduction. |
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
2 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| description The split of ProteinGym substitution DMS assays that ProteinGym evaluated on. The upstream dataset release is not catalogued here, so no claim is made that this matches its original splits. Context-only references | ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design No field-specific location recorded Version: Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| name ProteinGym substitution DMS assays (ProteinGym split) Context-only references | ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design No field-specific location recorded Version: Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
Sources and history
Release 2026-09-17-134cd1815de8 · Record review: source checked
1 source records and release history
- ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Original source · Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256
Technical metadata and extraction receipts
Stable ID: proteingym-dataset-proteingym-substitution-dms-assays
- areas
- proteins-complexes
- missing metadata
- version: unreported; url: unextracted
Related records
- dataset: CARP (640M) on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: CARP (640M) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: CARP (640M) on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: CARP (640M) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: CARP (640M) on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: DeepSequence (ensemble) on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: DeepSequence (ensemble) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: DeepSequence (ensemble) on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: DeepSequence (ensemble) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: DeepSequence (ensemble) on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: Embed. ESM-1v on ProteinGym SUP-SUB-MSE-AVG: Supervised substitutions, average over splits, MSE
- dataset: Embed. ESM-1v on ProteinGym SUP-SUB-MSE-CONTIG: Supervised substitutions, contiguous split, MSE
- dataset: Embed. ESM-1v on ProteinGym SUP-SUB-MSE-MOD: Supervised substitutions, modulo split, MSE
- dataset: Embed. ESM-1v on ProteinGym SUP-SUB-MSE-RAND: Supervised substitutions, random split, MSE
- dataset: Embed. ESM-1v on ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman
- dataset: Embed. ESM-1v on ProteinGym SUP-SUB-SPEARMAN-CONTIG: Supervised substitutions, contiguous split, Spearman
- dataset: Embed. ESM-1v on ProteinGym SUP-SUB-SPEARMAN-MOD: Supervised substitutions, modulo split, Spearman
- dataset: Embed. ESM-1v on ProteinGym SUP-SUB-SPEARMAN-RAND: Supervised substitutions, random split, Spearman
- dataset: Embed. MSAT on ProteinGym SUP-SUB-MSE-AVG: Supervised substitutions, average over splits, MSE
- dataset: Embed. MSAT on ProteinGym SUP-SUB-MSE-CONTIG: Supervised substitutions, contiguous split, MSE
- dataset: Embed. MSAT on ProteinGym SUP-SUB-MSE-MOD: Supervised substitutions, modulo split, MSE
- dataset: Embed. MSAT on ProteinGym SUP-SUB-MSE-RAND: Supervised substitutions, random split, MSE
- dataset: Embed. MSAT on ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman
- dataset: Embed. MSAT on ProteinGym SUP-SUB-SPEARMAN-CONTIG: Supervised substitutions, contiguous split, Spearman
- dataset: Embed. MSAT on ProteinGym SUP-SUB-SPEARMAN-MOD: Supervised substitutions, modulo split, Spearman
- dataset: Embed. MSAT on ProteinGym SUP-SUB-SPEARMAN-RAND: Supervised substitutions, random split, Spearman
- dataset: Embed. Tranception on ProteinGym SUP-SUB-MSE-AVG: Supervised substitutions, average over splits, MSE
- dataset: Embed. Tranception on ProteinGym SUP-SUB-MSE-CONTIG: Supervised substitutions, contiguous split, MSE
- dataset: Embed. Tranception on ProteinGym SUP-SUB-MSE-MOD: Supervised substitutions, modulo split, MSE
- dataset: Embed. Tranception on ProteinGym SUP-SUB-MSE-RAND: Supervised substitutions, random split, MSE
- dataset: Embed. Tranception on ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman
- dataset: Embed. Tranception on ProteinGym SUP-SUB-SPEARMAN-CONTIG: Supervised substitutions, contiguous split, Spearman
- dataset: Embed. Tranception on ProteinGym SUP-SUB-SPEARMAN-MOD: Supervised substitutions, modulo split, Spearman
- dataset: Embed. Tranception on ProteinGym SUP-SUB-SPEARMAN-RAND: Supervised substitutions, random split, Spearman
- dataset: ESM-1b on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: ESM-1b on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: ESM-1b on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: ESM-1b on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: ESM-1b on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: ESM-1v (ensemble) on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: ESM-1v (ensemble) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: ESM-1v (ensemble) on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: ESM-1v (ensemble) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: ESM-1v (ensemble) on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: ESM-2 (15B) on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: ESM-2 (15B) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: ESM-2 (15B) on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: ESM-2 (15B) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: ESM-2 (15B) on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: ESM-IF1 on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: ESM-IF1 on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: ESM-IF1 on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: ESM-IF1 on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: ESM-IF1 on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: EVE (ensemble) on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: EVE (ensemble) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: EVE (ensemble) on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: EVE (ensemble) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: EVE (ensemble) on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: EVmutation on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: EVmutation on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: EVmutation on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: EVmutation on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: EVmutation on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: GEMME on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: GEMME on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: GEMME on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: GEMME on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: GEMME on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: MIF-ST on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: MIF-ST on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: MIF-ST on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: MIF-ST on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: MIF-ST on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: MSA Transformer (ensemble) on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: MSA Transformer (ensemble) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: MSA Transformer (ensemble) on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: MSA Transformer (ensemble) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: MSA Transformer (ensemble) on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: NPT ProteinNPT on ProteinGym SUP-SUB-MSE-AVG: Supervised substitutions, average over splits, MSE
- dataset: NPT ProteinNPT on ProteinGym SUP-SUB-MSE-CONTIG: Supervised substitutions, contiguous split, MSE
- dataset: NPT ProteinNPT on ProteinGym SUP-SUB-MSE-MOD: Supervised substitutions, modulo split, MSE
- dataset: NPT ProteinNPT on ProteinGym SUP-SUB-MSE-RAND: Supervised substitutions, random split, MSE
- dataset: NPT ProteinNPT on ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman
- dataset: NPT ProteinNPT on ProteinGym SUP-SUB-SPEARMAN-CONTIG: Supervised substitutions, contiguous split, Spearman
- dataset: NPT ProteinNPT on ProteinGym SUP-SUB-SPEARMAN-MOD: Supervised substitutions, modulo split, Spearman
- dataset: NPT ProteinNPT on ProteinGym SUP-SUB-SPEARMAN-RAND: Supervised substitutions, random split, Spearman
- dataset: OHE DeepSequence on ProteinGym SUP-SUB-MSE-AVG: Supervised substitutions, average over splits, MSE
- dataset: OHE DeepSequence on ProteinGym SUP-SUB-MSE-CONTIG: Supervised substitutions, contiguous split, MSE
- dataset: OHE DeepSequence on ProteinGym SUP-SUB-MSE-MOD: Supervised substitutions, modulo split, MSE
- dataset: OHE DeepSequence on ProteinGym SUP-SUB-MSE-RAND: Supervised substitutions, random split, MSE
- dataset: OHE DeepSequence on ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman
- dataset: OHE DeepSequence on ProteinGym SUP-SUB-SPEARMAN-CONTIG: Supervised substitutions, contiguous split, Spearman
- dataset: OHE DeepSequence on ProteinGym SUP-SUB-SPEARMAN-MOD: Supervised substitutions, modulo split, Spearman
- dataset: OHE DeepSequence on ProteinGym SUP-SUB-SPEARMAN-RAND: Supervised substitutions, random split, Spearman
- dataset: OHE ESM-1v on ProteinGym SUP-SUB-MSE-AVG: Supervised substitutions, average over splits, MSE
- dataset: OHE ESM-1v on ProteinGym SUP-SUB-MSE-CONTIG: Supervised substitutions, contiguous split, MSE
- dataset: OHE ESM-1v on ProteinGym SUP-SUB-MSE-MOD: Supervised substitutions, modulo split, MSE
- dataset: OHE ESM-1v on ProteinGym SUP-SUB-MSE-RAND: Supervised substitutions, random split, MSE
- dataset: OHE ESM-1v on ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman
- dataset: OHE ESM-1v on ProteinGym SUP-SUB-SPEARMAN-CONTIG: Supervised substitutions, contiguous split, Spearman
- dataset: OHE ESM-1v on ProteinGym SUP-SUB-SPEARMAN-MOD: Supervised substitutions, modulo split, Spearman
- dataset: OHE ESM-1v on ProteinGym SUP-SUB-SPEARMAN-RAND: Supervised substitutions, random split, Spearman
- dataset: OHE MSAT on ProteinGym SUP-SUB-MSE-AVG: Supervised substitutions, average over splits, MSE
- dataset: OHE MSAT on ProteinGym SUP-SUB-MSE-CONTIG: Supervised substitutions, contiguous split, MSE
- dataset: OHE MSAT on ProteinGym SUP-SUB-MSE-MOD: Supervised substitutions, modulo split, MSE
- dataset: OHE MSAT on ProteinGym SUP-SUB-MSE-RAND: Supervised substitutions, random split, MSE
- dataset: OHE MSAT on ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman
- dataset: OHE MSAT on ProteinGym SUP-SUB-SPEARMAN-CONTIG: Supervised substitutions, contiguous split, Spearman
- dataset: OHE MSAT on ProteinGym SUP-SUB-SPEARMAN-MOD: Supervised substitutions, modulo split, Spearman
- dataset: OHE MSAT on ProteinGym SUP-SUB-SPEARMAN-RAND: Supervised substitutions, random split, Spearman
- dataset: OHE None on ProteinGym SUP-SUB-MSE-AVG: Supervised substitutions, average over splits, MSE
- dataset: OHE None on ProteinGym SUP-SUB-MSE-CONTIG: Supervised substitutions, contiguous split, MSE
- dataset: OHE None on ProteinGym SUP-SUB-MSE-MOD: Supervised substitutions, modulo split, MSE
- dataset: OHE None on ProteinGym SUP-SUB-MSE-RAND: Supervised substitutions, random split, MSE
- dataset: OHE None on ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman
- dataset: OHE None on ProteinGym SUP-SUB-SPEARMAN-CONTIG: Supervised substitutions, contiguous split, Spearman
- dataset: OHE None on ProteinGym SUP-SUB-SPEARMAN-MOD: Supervised substitutions, modulo split, Spearman
- dataset: OHE None on ProteinGym SUP-SUB-SPEARMAN-RAND: Supervised substitutions, random split, Spearman
- dataset: OHE TranceptEVE on ProteinGym SUP-SUB-MSE-AVG: Supervised substitutions, average over splits, MSE
- dataset: OHE TranceptEVE on ProteinGym SUP-SUB-MSE-CONTIG: Supervised substitutions, contiguous split, MSE
- dataset: OHE TranceptEVE on ProteinGym SUP-SUB-MSE-MOD: Supervised substitutions, modulo split, MSE
- dataset: OHE TranceptEVE on ProteinGym SUP-SUB-MSE-RAND: Supervised substitutions, random split, MSE
- dataset: OHE TranceptEVE on ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman
- dataset: OHE TranceptEVE on ProteinGym SUP-SUB-SPEARMAN-CONTIG: Supervised substitutions, contiguous split, Spearman
- dataset: OHE TranceptEVE on ProteinGym SUP-SUB-SPEARMAN-MOD: Supervised substitutions, modulo split, Spearman
- dataset: OHE TranceptEVE on ProteinGym SUP-SUB-SPEARMAN-RAND: Supervised substitutions, random split, Spearman
- dataset: OHE Tranception on ProteinGym SUP-SUB-MSE-AVG: Supervised substitutions, average over splits, MSE
- dataset: OHE Tranception on ProteinGym SUP-SUB-MSE-CONTIG: Supervised substitutions, contiguous split, MSE
- dataset: OHE Tranception on ProteinGym SUP-SUB-MSE-MOD: Supervised substitutions, modulo split, MSE
- dataset: OHE Tranception on ProteinGym SUP-SUB-MSE-RAND: Supervised substitutions, random split, MSE
- dataset: OHE Tranception on ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman
- dataset: OHE Tranception on ProteinGym SUP-SUB-SPEARMAN-CONTIG: Supervised substitutions, contiguous split, Spearman
- dataset: OHE Tranception on ProteinGym SUP-SUB-SPEARMAN-MOD: Supervised substitutions, modulo split, Spearman
- dataset: OHE Tranception on ProteinGym SUP-SUB-SPEARMAN-RAND: Supervised substitutions, random split, Spearman
- dataset: ProGen2 XL on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: ProGen2 XL on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: ProGen2 XL on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: ProGen2 XL on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: ProGen2 XL on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: ProteinMPNN on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: ProteinMPNN on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: ProteinMPNN on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: ProteinMPNN on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: ProteinMPNN on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: RITA XL on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: RITA XL on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: RITA XL on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: RITA XL on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: RITA XL on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: Site-Independent on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: Site-Independent on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: Site-Independent on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: Site-Independent on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: Site-Independent on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: TranceptEVE L on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: TranceptEVE L on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: TranceptEVE L on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: TranceptEVE L on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: TranceptEVE L on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: Tranception L on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: Tranception L on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: Tranception L on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: Tranception L on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: Tranception L on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: UniRep evotuned on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: UniRep evotuned on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: UniRep evotuned on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: UniRep evotuned on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: UniRep evotuned on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: UniRep on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: UniRep on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: UniRep on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: UniRep on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: UniRep on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: VESPA on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: VESPA on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: VESPA on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: VESPA on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: VESPA on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
- dataset: WaveNet on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- dataset: WaveNet on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- dataset: WaveNet on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- dataset: WaveNet on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- dataset: WaveNet on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman