Regulatory-element identification tests whether DNA models distinguish ENCODE regulatory sequences from composition-matched controls.
Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Datasets | ENCODE candidate cis-regulatory elements paired with synthetic dinucleotide-shuffled controls.SourcesDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 |
| Splits | Test chromosomes are 5, 10, 14, 18, 20 and 22; validation uses 6 and 21; other chromosomes form training data. Supervised checkpoints are selected by validation loss.SourcesDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 |
| Metrics | Zero-shot accuracy measures which member of a matched pair receives higher sequence likelihood. Supervised absolute classification accuracy and paired ranking accuracy are separate quantities.SourcesDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 |
| Baselines | Six DNA language-model families are compared in zero-shot, frozen-embedding probing and LoRA fine-tuning settings; a supervised CNN supplies an ab-initio reference.SourcesDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 |
| Leakage controls | Chromosome holdout separates supervised fitting from testing. Synthetic negatives preserve dinucleotide composition; pretraining sequence overlap is not resolved by chromosome splitting.SourcesDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 |
| Uncertainty | The appendix reports a one-sided Wilcoxon rank-sum test for regulatory versus control likelihoods.SourcesDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 |
| Entity type | Paper-specific computational evaluation protocol.SourcesDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 |
| Organisms | Human.SourcesDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 |
| Assays | ENCODE regulatory-element annotations with synthetic sequence controls.SourcesDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 |
| Allowed inputs | DNA sequence, with matched controls preserving dinucleotide composition.SourcesDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 |
| Adaptation | Zero-shot likelihood comparison, frozen-model probing, LoRA fine-tuning and supervised CNN training are distinct regimes.SourcesDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 |
Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.
ENCODE candidate cis-regulatory elements paired with synthetic dinucleotide-shuffled controls. Test chromosomes are 5, 10, 14, 18, 20 and 22; validation uses 6 and 21; other chromosomes form training data. Supervised checkpoints are selected by validation loss. Zero-shot accuracy measures which member of a matched pair receives higher sequence likelihood. Supervised absolute classification accuracy and paired ranking accuracy are separate quantities. Six DNA language-model families are compared in zero-shot, frozen-embedding probing and LoRA fine-tuning settings; a supervised CNN supplies an ab-initio reference. Chromosome holdout separates supervised fitting from testing. Synthetic negatives preserve dinucleotide composition; pretraining sequence overlap is not resolved by chromosome splitting. The appendix reports a one-sided Wilcoxon rank-sum test for regulatory versus control likelihoods.
Each evaluation records what was tested and under which conditions.
Release 2026-09-17-d277315f7d76 · 1 evaluation · 1 metric row. Different protocols are not a single leaderboard.
| Metric and finding | Coverage and uncertainty | Evidence |
|---|---|---|
| DNABERT-2: regulatory element identification Configuration: DNABERT-2Task: regulatory element identificationDataset: DART-Eval cCREs versus matched shuffled controls zero-shot likelihood ranking: higher likelihood for cCRE than matched control Independent external evaluation · Evaluation metadata: needs review | ||
| 0.876 accuracy Unit: fraction · Direction: unknown | Uncertainty: not reported in legacy extract Scored: Not reported · Eligible: Not reported | source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 3 (PDF page 5), DNABERT-2 row, Zero-Shot Accuracy column Source checking is not independent reproduction. |
Last literature check: 2026-09-17. Dated primary-source discovery and protocol/table screening. Source checking does not mean experimental reproduction. Only separately extracted and independently reviewed numeric batches are publishable.
| Paper or primary resource | Version | Reference |
|---|---|---|
| DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA | NeurIPS 2024 Datasets and Benchmarks Track proceedings | Read source |
primary comparison tables located
Task-specific computational methodology and field context checked in the cited primary-source artifact. Source-backed fields, inapplicable evaluator dimensions and unresolved details are distinguished. Numerical results were not reproduced.
Stable record: reported-task-cdbee1c9285568Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
18 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Diagram caption Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol. Individual claims | DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 Version: NeurIPS 2024 Datasets and Benchmarks Track proceedings | source checked automated source review · 2026-09-16 Audit detailsTask-specific computational methodology and field context checked in the cited primary-source artifact. Source-backed fields, inapplicable evaluator dimensions and unresolved details are distinguished. Numerical results were not reproduced. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram steps ["Input: DNA sequence, with matched controls preserving dinucleotide composition.","Evaluation: Zero-shot likelihood comparison, frozen-model probing, LoRA fine-tuning and supervised CNN training are distinct regimes.","Readout: Zero-shot accuracy measures which member of a matched pair receives higher sequence likelihood. Supervised absolute classification accuracy and paired ranking accuracy are separate quantities."] Individual claims | DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 Version: NeurIPS 2024 Datasets and Benchmarks Track proceedings | source checked automated source review · 2026-09-16 Audit detailsTask-specific computational methodology and field context checked in the cited primary-source artifact. Source-backed fields, inapplicable evaluator dimensions and unresolved details are distinguished. Numerical results were not reproduced. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Computational evaluation flow Individual claims | DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 Version: NeurIPS 2024 Datasets and Benchmarks Track proceedings | source checked automated source review · 2026-09-16 Audit detailsTask-specific computational methodology and field context checked in the cited primary-source artifact. Source-backed fields, inapplicable evaluator dimensions and unresolved details are distinguished. Numerical results were not reproduced. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Datasets ENCODE candidate cis-regulatory elements paired with synthetic dinucleotide-shuffled controls. Individual claims | DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 Version: NeurIPS 2024 Datasets and Benchmarks Track proceedings | source checked automated source review · 2026-09-16 Audit detailsTask-specific computational methodology and field context checked in the cited primary-source artifact. Source-backed fields, inapplicable evaluator dimensions and unresolved details are distinguished. Numerical results were not reproduced. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Splits Test chromosomes are 5, 10, 14, 18, 20 and 22; validation uses 6 and 21; other chromosomes form training data. Supervised checkpoints are selected by validation loss. Individual claims | DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 Version: NeurIPS 2024 Datasets and Benchmarks Track proceedings | source checked automated source review · 2026-09-16 Audit detailsTask-specific computational methodology and field context checked in the cited primary-source artifact. Source-backed fields, inapplicable evaluator dimensions and unresolved details are distinguished. Numerical results were not reproduced. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Adaptation Zero-shot likelihood comparison, frozen-model probing, LoRA fine-tuning and supervised CNN training are distinct regimes. Individual claims | DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 Version: NeurIPS 2024 Datasets and Benchmarks Track proceedings | source checked automated source review · 2026-09-16 Audit detailsTask-specific computational methodology and field context checked in the cited primary-source artifact. Source-backed fields, inapplicable evaluator dimensions and unresolved details are distinguished. Numerical results were not reproduced. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Metrics Zero-shot accuracy measures which member of a matched pair receives higher sequence likelihood. Supervised absolute classification accuracy and paired ranking accuracy are separate quantities. Individual claims | DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 Version: NeurIPS 2024 Datasets and Benchmarks Track proceedings | source checked automated source review · 2026-09-16 Audit detailsTask-specific computational methodology and field context checked in the cited primary-source artifact. Source-backed fields, inapplicable evaluator dimensions and unresolved details are distinguished. Numerical results were not reproduced. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Baselines Six DNA language-model families are compared in zero-shot, frozen-embedding probing and LoRA fine-tuning settings; a supervised CNN supplies an ab-initio reference. Individual claims | DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 Version: NeurIPS 2024 Datasets and Benchmarks Track proceedings | source checked automated source review · 2026-09-16 Audit detailsTask-specific computational methodology and field context checked in the cited primary-source artifact. Source-backed fields, inapplicable evaluator dimensions and unresolved details are distinguished. Numerical results were not reproduced. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Leakage controls Chromosome holdout separates supervised fitting from testing. Synthetic negatives preserve dinucleotide composition; pretraining sequence overlap is not resolved by chromosome splitting. Individual claims | DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 Version: NeurIPS 2024 Datasets and Benchmarks Track proceedings | source checked automated source review · 2026-09-16 Audit detailsTask-specific computational methodology and field context checked in the cited primary-source artifact. Source-backed fields, inapplicable evaluator dimensions and unresolved details are distinguished. Numerical results were not reproduced. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Uncertainty The appendix reports a one-sided Wilcoxon rank-sum test for regulatory versus control likelihoods. Individual claims | DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA Paper §3, §4.1, Table 3; Appendices D.2–D.3 and E.1 Version: NeurIPS 2024 Datasets and Benchmarks Track proceedings | source checked automated source review · 2026-09-16 Audit detailsTask-specific computational methodology and field context checked in the cited primary-source artifact. Source-backed fields, inapplicable evaluator dimensions and unresolved details are distinguished. Numerical results were not reproduced. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Release 2026-09-17-d277315f7d76 · Record review: needs review
Stable ID: reported-task-cdbee1c9285568