The TAPE remote protein-homology classification task evaluates a trained protein representation.
Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Datasets | SCOP 1.75 domains with fold labels and held-out superfamilies.Sourcestape primary benchmark evidence · Appendix A.1 |
| Splits | SCOP 1.75 protein domains are grouped by evolutionary hierarchy; entire superfamilies are held out for fold-level classification.Sourcestape primary benchmark evidence · Section 3 task definitions; Appendix A.1.1–A.1.5 |
| Metrics | Top-1 class accuracy.Sourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
| Baselines | Task leaderboard comparisons include Transformer, LSTM, UniRep, ResNet, Bepler and one-hot baselines; alignment-augmented references appear where applicable.Sourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
| Leakage controls | Holding out superfamilies tests remote homologues without transferring examples from the same superfamily between train and test.Sourcestape primary benchmark evidence · Section 3 task definitions; Appendix A.1.1–A.1.5 |
| Uncertainty | The original paper reports point estimates in its task result tables. Methods and task appendices do not define a suite-wide repeated-seed or bootstrap interval; individual later evaluations must supply their own uncertainty. · Not reported in inspected sourcesSourcestape primary benchmark evidence · Section 3; Appendix A.1; Tables 1–2 |
| Entity type | Constituent benchmark task: TAPE Remote Homology DetectionSourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
| Organisms | SCOP protein-domain collection across organisms; the evaluated label is structural fold, not species.Sourcestape primary benchmark evidence · Section 3 task definitions; Appendix A.1.1–A.1.5 |
| Assays | Protein structure/homology annotations and experimental fluorescence/stability measurements.Sourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
| Allowed inputs | Protein sequence with a remote-homology class.Sourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
| Adaptation | Unsupervised pretraining followed by supervised downstream training; the README warns that downstream hyperparameters require task-specific tuning.Sourcessonglab-cal/tape official source · Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references |
Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions.
SCOP 1.75 protein domains are grouped by evolutionary hierarchy; entire superfamilies are held out for fold-level classification. Holding out superfamilies tests remote homologues without transferring examples from the same superfamily between train and test.
Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.
These source-backed links do not make different protocols or scores interchangeable.
Each evaluation records what was tested and under which conditions.
Explore the results reported under one evaluation protocol. Each figure keeps its source, dataset and metric together; it is not a ranking across studies.
accuracy (fraction) · Higher values are better for this metric.
Held-out evolutionary groups; fold-level classification into 1,195 folds
Source order is preserved. Plotted marks show point estimates; uncertainty, where reported, is retained in the printed values and table. Differences do not establish statistical significance.
tape primary benchmark evidence · Results on downstream supervised tasks; Table2,p.7,row1(No pretraining/Transformer),columnRemote homology; Table2,p.7,row2(No pretraining/LSTM),columnRemote homology; Table2,p.7,row3(No pretraining/ResNet),columnRemote homology; Table2,p.7,row4(Self-supervised pretraining/Transformer),columnRemote homology; Table2,p.7,row5(Self-supervised pretraining/LSTM),columnRemote homology; Table2,p.7,row6(Self-supervised pretraining/ResNet),columnRemote homology; Table2,p.7,row7(Supervised pretraining/Bepler supervised LSTM),columnRemote homology; Table2,p.7,row8(Self-supervised pretraining/UniRep mLSTM),columnRemote homology; Table2,p.7,row9(One-hot baseline/One-hot),columnRemote homology; Table2,p.7,row10(Task-specific alignment/HMM baseline/Alignment),columnRemote homology| Tested entity | Printed value | Uncertainty | Evidence |
|---|---|---|---|
| Transformer (no pretraining) · Configuration | 0.09 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row1(No pretraining/Transformer),columnRemote homology |
| LSTM (no pretraining) · Configuration | 0.12 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row2(No pretraining/LSTM),columnRemote homology |
| ResNet (no pretraining) · Configuration | 0.10 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row3(No pretraining/ResNet),columnRemote homology |
| TAPE Transformer · Model | 0.21 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row4(Self-supervised pretraining/Transformer),columnRemote homology |
| TAPE LSTM · Model | 0.26 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row5(Self-supervised pretraining/LSTM),columnRemote homology |
| TAPE ResNet · Model | 0.17 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row6(Self-supervised pretraining/ResNet),columnRemote homology |
| TAPE Bepler · Model | 0.17 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row7(Supervised pretraining/Bepler supervised LSTM),columnRemote homology |
| TAPE Unirep · Model | 0.23 fraction | Not reported | Independent external evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row8(Self-supervised pretraining/UniRep mLSTM),columnRemote homology |
| TAPE One Hot · Method | 0.09 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row9(One-hot baseline/One-hot),columnRemote homology |
| Alignment · Configuration | 0.09 fraction | Not reported | Author-reported evaluation · source checkedtape primary benchmark evidence · Table2,p.7,row10(Task-specific alignment/HMM baseline/Alignment),columnRemote homology |
Source transcription and grouping reviewed by automated source review on 2026-09-17. These experiments were not independently reproduced by rewire.
Release 2026-09-17-d277315f7d76 · 10 evaluations · 10 metric rows. Different protocols are not a single leaderboard.
| Metric and finding | Coverage and uncertainty | Evidence |
|---|---|---|
| TAPE ResNet: SCOP1.75 fold-level test Held-out evolutionary groups; fold-level classification into 1,195 folds Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.17 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row6(Self-supervised pretraining/ResNet),columnRemote homology Source checking is not independent reproduction. |
| TAPE Bepler: SCOP1.75 fold-level test Held-out evolutionary groups; fold-level classification into 1,195 folds Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.17 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row7(Supervised pretraining/Bepler supervised LSTM),columnRemote homology Source checking is not independent reproduction. |
| TAPE LSTM: SCOP1.75 fold-level test Held-out evolutionary groups; fold-level classification into 1,195 folds Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.26 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row5(Self-supervised pretraining/LSTM),columnRemote homology Source checking is not independent reproduction. |
| TAPE One Hot: SCOP1.75 fold-level test Held-out evolutionary groups; fold-level classification into 1,195 folds Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.09 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row9(One-hot baseline/One-hot),columnRemote homology Source checking is not independent reproduction. |
| Alignment: SCOP1.75 fold-level test Held-out evolutionary groups; fold-level classification into 1,195 folds Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.09 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row10(Task-specific alignment/HMM baseline/Alignment),columnRemote homology Source checking is not independent reproduction. |
| Transformer (no pretraining): SCOP1.75 fold-level test Configuration: Transformer (no pretraining)Task: TAPE Remote Homology DetectionDataset: SCOP1.75 fold-level test Held-out evolutionary groups; fold-level classification into 1,195 folds Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.09 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row1(No pretraining/Transformer),columnRemote homology Source checking is not independent reproduction. |
| TAPE Unirep: SCOP1.75 fold-level test Held-out evolutionary groups; fold-level classification into 1,195 folds Independent external evaluation · Evaluation metadata: needs review | ||
| 0.23 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row8(Self-supervised pretraining/UniRep mLSTM),columnRemote homology Source checking is not independent reproduction. |
| LSTM (no pretraining): SCOP1.75 fold-level test Configuration: LSTM (no pretraining)Task: TAPE Remote Homology DetectionDataset: SCOP1.75 fold-level test Held-out evolutionary groups; fold-level classification into 1,195 folds Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.12 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row2(No pretraining/LSTM),columnRemote homology Source checking is not independent reproduction. |
| TAPE Transformer: SCOP1.75 fold-level test Held-out evolutionary groups; fold-level classification into 1,195 folds Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.21 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row4(Self-supervised pretraining/Transformer),columnRemote homology Source checking is not independent reproduction. |
| ResNet (no pretraining): SCOP1.75 fold-level test Configuration: ResNet (no pretraining)Task: TAPE Remote Homology DetectionDataset: SCOP1.75 fold-level test Held-out evolutionary groups; fold-level classification into 1,195 folds Author-reported evaluation · Evaluation metadata: needs review | ||
| 0.10 accuracy Unit: fraction · Direction: higher | Uncertainty: unreported Scored: Not reported · Eligible: Not reported | source checkedtape primary benchmark evidence · Table2,p.7,row3(No pretraining/ResNet),columnRemote homology Source checking is not independent reproduction. |
Last literature check: 2026-09-17. Dated primary-source discovery and protocol/table screening. Source checking does not mean experimental reproduction. Only separately extracted and independently reviewed numeric batches are publishable.
| Paper or primary resource | Version | Reference |
|---|---|---|
| Evaluating Protein Transfer Learning with TAPE | 1906.08230v1 | Read source |
complete tables extracted
Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.
Stable record: discovery-benchmark-tape-remote-homology-detectionApplicability is distinct from a completed evaluation.
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
23 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Diagram caption Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions. Individual claims | tape primary benchmark evidence Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1906.08230v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram caption Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions. Individual claims | songlab-cal/tape official source Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 6d345c2b2bbf52cd32cf179325c222afd92aec7e | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram steps ["Allowed inputs: Protein sequence with a remote-homology class.","Splits: SCOP 1.75 protein domains are grouped by evolutionary hierarchy; entire superfamilies are held out for fold-level classification.","Metrics: Top-1 class accuracy."] Individual claims | tape primary benchmark evidence Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1906.08230v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram steps ["Allowed inputs: Protein sequence with a remote-homology class.","Splits: SCOP 1.75 protein domains are grouped by evolutionary hierarchy; entire superfamilies are held out for fold-level classification.","Metrics: Top-1 class accuracy."] Individual claims | songlab-cal/tape official source Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 6d345c2b2bbf52cd32cf179325c222afd92aec7e | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Evaluation procedure Individual claims | tape primary benchmark evidence Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1906.08230v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Evaluation procedure Individual claims | songlab-cal/tape official source Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references; Section 3 task definitions; Appendix A.1.1–A.1.5 Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 6d345c2b2bbf52cd32cf179325c222afd92aec7e | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Datasets SCOP 1.75 domains with fold labels and held-out superfamilies. Individual claims | tape primary benchmark evidence Appendix A.1 Version: 1906.08230v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Splits SCOP 1.75 protein domains are grouped by evolutionary hierarchy; entire superfamilies are held out for fold-level classification. Individual claims | tape primary benchmark evidence Section 3 task definitions; Appendix A.1.1–A.1.5 Version: 1906.08230v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Adaptation Unsupervised pretraining followed by supervised downstream training; the README warns that downstream hyperparameters require task-specific tuning. Individual claims | songlab-cal/tape official source Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references Version: 6d345c2b2bbf52cd32cf179325c222afd92aec7e | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Metrics Top-1 class accuracy. Individual claims | songlab-cal/tape official source Pinned README: compatibility notice; Evaluating a Downstream Model; Data; task leaderboards; README task-specific leaderboard and data-source references Version: 6d345c2b2bbf52cd32cf179325c222afd92aec7e | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Release 2026-09-17-d277315f7d76 · Record review: discovered
Stable ID: discovery-benchmark-tape-remote-homology-detection