The TAPE residue secondary-structure classification task evaluates a trained protein representation.
Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Datasets | Klausen training/validation proteins with CB513, CASP12 and TS115 structure-label test collections.Sourcestape primary benchmark evidence · Appendix A.1 |
| Splits | Training/validation and CB513, CASP12 and TS115 test proteins are filtered at 25% sequence identity.Sourcestape primary benchmark evidence · Section 3 task definitions; Appendix A.1.1–A.1.5 |
| Metrics | Three-class residue accuracy.Sourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
| Baselines | Task leaderboard comparisons include Transformer, LSTM, UniRep, ResNet, Bepler and one-hot baselines; alignment-augmented references appear where applicable.Sourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
| Leakage controls | Identity filtering excludes close train/test homologues; it is not a species holdout.Sourcestape primary benchmark evidence · Section 3 task definitions; Appendix A.1.1–A.1.5 |
| Uncertainty | The original paper reports point estimates in its task result tables. Methods and task appendices do not define a suite-wide repeated-seed or bootstrap interval; individual later evaluations must supply their own uncertainty. · Not reported in inspected sourcesSourcestape primary benchmark evidence · Section 3; Appendix A.1; Tables 1–2 |
| Entity type | Constituent benchmark task: TAPE Secondary StructureSourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
| Organisms | Protein structures from the mixed-organism Klausen/CB513/CASP12/TS115 collections.Sourcestape primary benchmark evidence · Section 3 task definitions; Appendix A.1.1–A.1.5 |
| Assays | Protein structure/homology annotations and experimental fluorescence/stability measurements.Sourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
| Allowed inputs | Protein amino-acid sequence with residue-level secondary-structure labels.Sourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
| Adaptation | Unsupervised pretraining followed by supervised downstream training; the README warns that downstream hyperparameters require task-specific tuning.Sourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions.
Training/validation and CB513, CASP12 and TS115 test proteins are filtered at 25% sequence identity. Identity filtering excludes close train/test homologues; it is not a species holdout.
Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.
These source-backed links do not make different protocols or scores interchangeable.
Each evaluation records what was tested and under which conditions.
Explore the results reported under one evaluation protocol. Each figure keeps its source, dataset and metric together; it is not a ranking across studies.
accuracy (fraction) · Higher values are better for this metric.
CB513 test set; train/test proteins filtered at 25% sequence identity
Source order is preserved. Plotted marks show point estimates; uncertainty, where reported, is retained in the printed values and table. Differences do not establish statistical significance.
tape primary benchmark evidence · Results on downstream supervised tasks; Table2,p.7,row1(No pretraining/Transformer),columnSecondary structure; Table2,p.7,row2(No pretraining/LSTM),columnSecondary structure; Table2,p.7,row3(No pretraining/ResNet),columnSecondary structure; Table2,p.7,row4(Self-supervised pretraining/Transformer),columnSecondary structure; Table2,p.7,row5(Self-supervised pretraining/LSTM),columnSecondary structure; Table2,p.7,row6(Self-supervised pretraining/ResNet),columnSecondary structure; Table2,p.7,row7(Supervised pretraining/Bepler supervised LSTM),columnSecondary structure; Table2,p.7,row8(Self-supervised pretraining/UniRep mLSTM),columnSecondary structure; Table2,p.7,row9(One-hot baseline/One-hot),columnSecondary structure; Table2,p.7,row10(Task-specific alignment/HMM baseline/Alignment),columnSecondary structure| Tested entity | Printed value | Uncertainty | Evidence |
|---|---|---|---|
| Transformer (no pretraining) · Configuration | 0.70 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row1(No pretraining/Transformer),columnSecondary structure |
| LSTM (no pretraining) · Configuration | 0.71 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row2(No pretraining/LSTM),columnSecondary structure |
| ResNet (no pretraining) · Configuration | 0.70 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row3(No pretraining/ResNet),columnSecondary structure |
| TAPE Transformer · Model | 0.73 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row4(Self-supervised pretraining/Transformer),columnSecondary structure |
| TAPE LSTM · Model | 0.75 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row5(Self-supervised pretraining/LSTM),columnSecondary structure |
| TAPE ResNet · Model | 0.75 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row6(Self-supervised pretraining/ResNet),columnSecondary structure |
| TAPE Bepler · Model | 0.73 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row7(Supervised pretraining/Bepler supervised LSTM),columnSecondary structure |
| TAPE Unirep · Model | 0.73 fraction | Not reported | Independent external evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row8(Self-supervised pretraining/UniRep mLSTM),columnSecondary structure |
| TAPE One Hot · Method | 0.69 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row9(One-hot baseline/One-hot),columnSecondary structure |
| Alignment · Configuration | 0.80 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row10(Task-specific alignment/HMM baseline/Alignment),columnSecondary structure |
Source transcription and grouping reviewed by automated source review on 2026-09-17. These experiments were not independently reproduced by rewire.
Release 2026-09-17-d277315f7d76 · 10 evaluations · 10 metric rows. Different protocols are not a single leaderboard.
| Metric and finding | Coverage and uncertainty | Evidence |
|---|---|---|
| TAPE Bepler: CB513 CB513 test set; train/test proteins filtered at 25% sequence identity Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.73 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row7(Supervised pretraining/Bepler supervised LSTM),columnSecondary structure Source checking is not independent reproduction. |
| Transformer (no pretraining): CB513 CB513 test set; train/test proteins filtered at 25% sequence identity Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.70 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row1(No pretraining/Transformer),columnSecondary structure Source checking is not independent reproduction. |
| Alignment: CB513 CB513 test set; train/test proteins filtered at 25% sequence identity Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.80 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row10(Task-specific alignment/HMM baseline/Alignment),columnSecondary structure Source checking is not independent reproduction. |
| TAPE Unirep: CB513 CB513 test set; train/test proteins filtered at 25% sequence identity Independent external evaluation · Evaluation metadata: needs review | ||
| 0.73 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row8(Self-supervised pretraining/UniRep mLSTM),columnSecondary structure Source checking is not independent reproduction. |
| ResNet (no pretraining): CB513 CB513 test set; train/test proteins filtered at 25% sequence identity Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.70 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row3(No pretraining/ResNet),columnSecondary structure Source checking is not independent reproduction. |
| LSTM (no pretraining): CB513 CB513 test set; train/test proteins filtered at 25% sequence identity Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.71 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row2(No pretraining/LSTM),columnSecondary structure Source checking is not independent reproduction. |
| TAPE Transformer: CB513 CB513 test set; train/test proteins filtered at 25% sequence identity Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.73 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row4(Self-supervised pretraining/Transformer),columnSecondary structure Source checking is not independent reproduction. |
| TAPE One Hot: CB513 CB513 test set; train/test proteins filtered at 25% sequence identity Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.69 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row9(One-hot baseline/One-hot),columnSecondary structure Source checking is not independent reproduction. |
| TAPE LSTM: CB513 CB513 test set; train/test proteins filtered at 25% sequence identity Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.75 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row5(Self-supervised pretraining/LSTM),columnSecondary structure Source checking is not independent reproduction. |
| TAPE ResNet: CB513 CB513 test set; train/test proteins filtered at 25% sequence identity Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.75 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row6(Self-supervised pretraining/ResNet),columnSecondary structure Source checking is not independent reproduction. |
Last literature check: 2026-09-17. Dated primary-source discovery and protocol/table screening. Source checking does not mean experimental reproduction. Only separately extracted and independently reviewed numeric batches are publishable.
| Paper or primary resource | Version | Reference |
|---|---|---|
| Evaluating Protein Transfer Learning with TAPE | 1906.08230v1 | Read source |
complete tables extracted
Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.
Stable record: discovery-benchmark-tape-secondary-structureTrace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
23 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Diagram caption Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions. Individual claims | tape primary benchmark evidence Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1906.08230v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram caption Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions. Individual claims | songlab-cal/tape official source Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 6d345c2b2bbf52cd32cf179325c222afd92aec7e | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram steps ["Allowed inputs: Protein amino-acid sequence with residue-level secondary-structure labels.","Splits: Training/validation and CB513, CASP12 and TS115 test proteins are filtered at 25% sequence identity.","Metrics: Three-class residue accuracy."] Individual claims | tape primary benchmark evidence Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1906.08230v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram steps ["Allowed inputs: Protein amino-acid sequence with residue-level secondary-structure labels.","Splits: Training/validation and CB513, CASP12 and TS115 test proteins are filtered at 25% sequence identity.","Metrics: Three-class residue accuracy."] Individual claims | songlab-cal/tape official source Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 6d345c2b2bbf52cd32cf179325c222afd92aec7e | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Evaluation procedure Individual claims | tape primary benchmark evidence Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1906.08230v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Evaluation procedure Individual claims | songlab-cal/tape official source Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 6d345c2b2bbf52cd32cf179325c222afd92aec7e | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Datasets Klausen training/validation proteins with CB513, CASP12 and TS115 structure-label test collections. Individual claims | tape primary benchmark evidence Appendix A.1 Version: 1906.08230v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Splits Training/validation and CB513, CASP12 and TS115 test proteins are filtered at 25% sequence identity. Individual claims | tape primary benchmark evidence Section 3 task definitions; Appendix A.1.1–A.1.5 Version: 1906.08230v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Adaptation Unsupervised pretraining followed by supervised downstream training; the README warns that downstream hyperparameters require task-specific tuning. Individual claims | songlab-cal/tape official source Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references Version: 6d345c2b2bbf52cd32cf179325c222afd92aec7e | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Metrics Three-class residue accuracy. Individual claims | songlab-cal/tape official source Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references Version: 6d345c2b2bbf52cd32cf179325c222afd92aec7e | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Release 2026-09-17-d277315f7d76 · Record review: discovered
Stable ID: discovery-benchmark-tape-secondary-structure