The TAPE protein stability prediction task evaluates a trained protein representation.
Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Datasets | Rocklin designed-protein stability measurements, using the original TAPE round-to-neighbourhood partition.Sourcestape primary benchmark evidence · Appendix A.1 |
| Splits | Training/validation use four rounds of designed-protein stability experiments; testing uses seventeen one-mutation neighbourhoods around selected promising proteins.Sourcestape primary benchmark evidence · Section 3 task definitions; Appendix A.1.1–A.1.5 |
| Metrics | Spearman rank correlation.Sourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
| Baselines | Task leaderboard comparisons include Transformer, LSTM, UniRep, ResNet, Bepler and one-hot baselines; alignment-augmented references appear where applicable.Sourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
| Leakage controls | Designed round-to-neighbourhood generalization is intentional; test variants can be close to parent proteins observed during training.Sourcestape primary benchmark evidence · Section 3 task definitions; Appendix A.1.1–A.1.5 |
| Uncertainty | The original paper reports point estimates in its task result tables. Methods and task appendices do not define a suite-wide repeated-seed or bootstrap interval; individual later evaluations must supply their own uncertainty. · Not reported in inspected sourcesSourcestape primary benchmark evidence · Section 3; Appendix A.1; Tables 1–2 |
| Entity type | Constituent benchmark task: TAPE StabilitySourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
| Organisms | Designed protein sequences measured by the Rocklin stability assay; a single natural organism label is inapplicable.Sourcestape primary benchmark evidence · Section 3 task definitions; Appendix A.1.1–A.1.5 |
| Assays | Protein structure/homology annotations and experimental fluorescence/stability measurements.Sourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
| Allowed inputs | Protein sequence with a measured stability target.Sourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
| Adaptation | Unsupervised pretraining followed by supervised downstream training; the README warns that downstream hyperparameters require task-specific tuning.Sourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions.
Training/validation use four rounds of designed-protein stability experiments; testing uses seventeen one-mutation neighbourhoods around selected promising proteins. Designed round-to-neighbourhood generalization is intentional; test variants can be close to parent proteins observed during training.
Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.
These source-backed links do not make different protocols or scores interchangeable.
Each evaluation records what was tested and under which conditions.
Explore the results reported under one evaluation protocol. Each figure keeps its source, dataset and metric together; it is not a ranking across studies.
Spearman's rho (correlation) · Higher values are better for this metric.
Test 17 one-mutation neighbourhoods around candidate proteins; train on broad experimental design rounds
Source order is preserved. 1 unavailable values are omitted from the plot and retained in the table. Plotted marks show point estimates; uncertainty, where reported, is retained in the printed values and table. Differences do not establish statistical significance.
tape primary benchmark evidence · Results on downstream supervised tasks; Table2,p.7,row1(No pretraining/Transformer),columnStability; Table2,p.7,row2(No pretraining/LSTM),columnStability; Table2,p.7,row3(No pretraining/ResNet),columnStability; Table2,p.7,row4(Self-supervised pretraining/Transformer),columnStability; Table2,p.7,row5(Self-supervised pretraining/LSTM),columnStability; Table2,p.7,row6(Self-supervised pretraining/ResNet),columnStability; Table2,p.7,row7(Supervised pretraining/Bepler supervised LSTM),columnStability; Table2,p.7,row8(Self-supervised pretraining/UniRep mLSTM),columnStability; Table2,p.7,row9(One-hot baseline/One-hot),columnStability; Table2,p.7,row10(Task-specific alignment/HMM baseline/Alignment),columnStability| Tested entity | Printed value | Uncertainty | Evidence |
|---|---|---|---|
| Transformer (no pretraining) · Configuration | -0.06 correlation | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row1(No pretraining/Transformer),columnStability |
| LSTM (no pretraining) · Configuration | 0.28 correlation | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row2(No pretraining/LSTM),columnStability |
| ResNet (no pretraining) · Configuration | 0.61 correlation | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row3(No pretraining/ResNet),columnStability |
| TAPE Transformer · Model | 0.73 correlation | Not reported | Author-reported evaluation · source checkedsonglab-cal/tape official source; tape primary benchmark evidence · README.md > Leaderboard > Stability; row Transformer; column Spearman's rho |
| TAPE LSTM · Model | 0.69 correlation | Not reported | Author-reported evaluation · source checkedsonglab-cal/tape official source; tape primary benchmark evidence · README.md > Leaderboard > Stability; row LSTM; column Spearman's rho |
| TAPE ResNet · Model | 0.73 correlation | Not reported | Author-reported evaluation · source checkedsonglab-cal/tape official source; tape primary benchmark evidence · README.md > Leaderboard > Stability; row ResNet; column Spearman's rho |
| TAPE Bepler · Model | 0.64 correlation | Not reported | Author-reported evaluation · source checkedsonglab-cal/tape official source; tape primary benchmark evidence · README.md > Leaderboard > Stability; row Bepler; column Spearman's rho |
| TAPE Unirep · Model | 0.73 correlation | Not reported | Independent external evaluation · source checkedsonglab-cal/tape official source; tape primary benchmark evidence · README.md > Leaderboard > Stability; row Unirep; column Spearman's rho |
| TAPE One Hot · Method | 0.19 correlation | Not reported | Author-reported evaluation · source checkedsonglab-cal/tape official source; tape primary benchmark evidence · README.md > Leaderboard > Stability; row One Hot; column Spearman's rho |
| Alignment · Configuration | N/A correlation | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row10(Task-specific alignment/HMM baseline/Alignment),columnStability |
Source transcription and grouping reviewed by automated source review on 2026-09-17. These experiments were not independently reproduced by rewire.
Release 2026-09-17-d277315f7d76 · 10 evaluations · 10 metric rows. Different protocols are not a single leaderboard.
| Metric and finding | Coverage and uncertainty | Evidence |
|---|---|---|
| TAPE Stability Bepler leaderboard evaluation Test 17 one-mutation neighbourhoods around candidate proteins; train on broad experimental design rounds Author-reported evaluation · Evaluation metadata: discovered | ||
| 0.64 Spearman's rho Unit: correlation · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedsonglab-cal/tape official source; tape primary benchmark evidence · README.md > Leaderboard > Stability; row Bepler; column Spearman's rho Source checking is not independent reproduction. |
| TAPE Stability LSTM leaderboard evaluation Test 17 one-mutation neighbourhoods around candidate proteins; train on broad experimental design rounds Author-reported evaluation · Evaluation metadata: discovered | ||
| 0.69 Spearman's rho Unit: correlation · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedsonglab-cal/tape official source; tape primary benchmark evidence · README.md > Leaderboard > Stability; row LSTM; column Spearman's rho Source checking is not independent reproduction. |
| TAPE Stability One Hot leaderboard evaluation Test 17 one-mutation neighbourhoods around candidate proteins; train on broad experimental design rounds Author-reported evaluation · Evaluation metadata: discovered | ||
| 0.19 Spearman's rho Unit: correlation · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedsonglab-cal/tape official source; tape primary benchmark evidence · README.md > Leaderboard > Stability; row One Hot; column Spearman's rho Source checking is not independent reproduction. |
| TAPE Stability ResNet leaderboard evaluation Test 17 one-mutation neighbourhoods around candidate proteins; train on broad experimental design rounds Author-reported evaluation · Evaluation metadata: discovered | ||
| 0.73 Spearman's rho Unit: correlation · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedsonglab-cal/tape official source; tape primary benchmark evidence · README.md > Leaderboard > Stability; row ResNet; column Spearman's rho Source checking is not independent reproduction. |
| TAPE Stability Transformer leaderboard evaluation Test 17 one-mutation neighbourhoods around candidate proteins; train on broad experimental design rounds Author-reported evaluation · Evaluation metadata: discovered | ||
| 0.73 Spearman's rho Unit: correlation · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedsonglab-cal/tape official source; tape primary benchmark evidence · README.md > Leaderboard > Stability; row Transformer; column Spearman's rho Source checking is not independent reproduction. |
| TAPE Stability Unirep leaderboard evaluation Test 17 one-mutation neighbourhoods around candidate proteins; train on broad experimental design rounds Independent external evaluation · Evaluation metadata: discovered | ||
| 0.73 Spearman's rho Unit: correlation · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedsonglab-cal/tape official source; tape primary benchmark evidence · README.md > Leaderboard > Stability; row Unirep; column Spearman's rho Source checking is not independent reproduction. |
| Alignment: Rocklin designed-protein stability Test 17 one-mutation neighbourhoods around candidate proteins; train on broad experimental design rounds Author-reported evaluation · Evaluation metadata: needs review | ||
| N/A Spearman's rho Unit: correlation · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row10(Task-specific alignment/HMM baseline/Alignment),columnStability Source checking is not independent reproduction. |
| LSTM (no pretraining): Rocklin designed-protein stability Test 17 one-mutation neighbourhoods around candidate proteins; train on broad experimental design rounds Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.28 Spearman's rho Unit: correlation · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row2(No pretraining/LSTM),columnStability Source checking is not independent reproduction. |
| Transformer (no pretraining): Rocklin designed-protein stability Configuration: Transformer (no pretraining)Task: TAPE StabilityDataset: TAPE Stability source dataset Test 17 one-mutation neighbourhoods around candidate proteins; train on broad experimental design rounds Author-reported evaluation · Evaluation metadata: needs review | ||
| -0.06 Spearman's rho Unit: correlation · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row1(No pretraining/Transformer),columnStability Source checking is not independent reproduction. |
| ResNet (no pretraining): Rocklin designed-protein stability Test 17 one-mutation neighbourhoods around candidate proteins; train on broad experimental design rounds Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.61 Spearman's rho Unit: correlation · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row3(No pretraining/ResNet),columnStability Source checking is not independent reproduction. |
Last literature check: 2026-09-17. Dated primary-source discovery and protocol/table screening. Source checking does not mean experimental reproduction. Only separately extracted and independently reviewed numeric batches are publishable.
| Paper or primary resource | Version | Reference |
|---|---|---|
| Evaluating Protein Transfer Learning with TAPE | 1906.08230v1 | Read source |
complete tables extracted
Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.
Stable record: discovery-benchmark-tape-stabilityTrace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
23 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Diagram caption Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions. Individual claims | tape primary benchmark evidence Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1906.08230v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram caption Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions. Individual claims | songlab-cal/tape official source Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 6d345c2b2bbf52cd32cf179325c222afd92aec7e | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram steps ["Allowed inputs: Protein sequence with a measured stability target.","Splits: Training/validation use four rounds of designed-protein stability experiments; testing uses seventeen one-mutation neighbourhoods around selected promising proteins.","Metrics: Spearman rank correlation."] Individual claims | tape primary benchmark evidence Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1906.08230v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram steps ["Allowed inputs: Protein sequence with a measured stability target.","Splits: Training/validation use four rounds of designed-protein stability experiments; testing uses seventeen one-mutation neighbourhoods around selected promising proteins.","Metrics: Spearman rank correlation."] Individual claims | songlab-cal/tape official source Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 6d345c2b2bbf52cd32cf179325c222afd92aec7e | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Evaluation procedure Individual claims | tape primary benchmark evidence Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1906.08230v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Evaluation procedure Individual claims | songlab-cal/tape official source Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 6d345c2b2bbf52cd32cf179325c222afd92aec7e | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Datasets Rocklin designed-protein stability measurements, using the original TAPE round-to-neighbourhood partition. Individual claims | tape primary benchmark evidence Appendix A.1 Version: 1906.08230v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Splits Training/validation use four rounds of designed-protein stability experiments; testing uses seventeen one-mutation neighbourhoods around selected promising proteins. Individual claims | tape primary benchmark evidence Section 3 task definitions; Appendix A.1.1–A.1.5 Version: 1906.08230v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Adaptation Unsupervised pretraining followed by supervised downstream training; the README warns that downstream hyperparameters require task-specific tuning. Individual claims | songlab-cal/tape official source Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references Version: 6d345c2b2bbf52cd32cf179325c222afd92aec7e | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Metrics Spearman rank correlation. Individual claims | songlab-cal/tape official source Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references Version: 6d345c2b2bbf52cd32cf179325c222afd92aec7e | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Release 2026-09-17-d277315f7d76 · Record review: discovered
Stable ID: discovery-benchmark-tape-stability