Task

DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC

Variant scoring on DNase QTLs in Yoruban LCLs, AUROC. Scored with AUROC on DNase QTLs in Yoruban LCLs. Score the effect of a variant on chromatin accessibility, against the measured QTL call.

22 evaluations · 22 metric rows

Overview

Variant scoring on DNase QTLs in Yoruban LCLs, AUROC. Scored with AUROC on DNase QTLs in Yoruban LCLs. Score the effect of a variant on chromatin accessibility, against the measured QTL call.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

Evaluation design

Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.

Benchmarks

DART-Eval

These source-backed links do not make different protocols or scores interchangeable.

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Explore all linked results

Run instructions

No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.

Published comparisons

Explore the results reported under one evaluation protocol. Each figure keeps its source, dataset and metric together; it is not a ranking across studies. The pooled view gathers every source table that reports the same metric and names what it does not hold constant.

Comparison and metric

DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC

auroc (fraction) · Higher values are better for this metric.

Every method DART-Eval reports on Variant scoring on DNase QTLs in Yoruban LCLs, AUROC, scored with AUROC on DNase QTLs in Yoruban LCLs.

Evaluation protocol · DNase QTLs in Yoruban LCLs (DART-Eval split)

Caduceus (zero-shot likelihood) · Configuration · Author-reported evaluation0.443
Caduceus (zero-shot embedding) · Configuration · Author-reported evaluation0.508
Caduceus (probed) · Configuration · Author-reported evaluation0.490
Caduceus (fine-tuned) · Configuration · Author-reported evaluation0.666
DNABERT-2 (zero-shot embedding) · Configuration · Author-reported evaluation0.505
DNABERT-2 (probed) · Configuration · Author-reported evaluation0.476
DNABERT-2 (fine-tuned) · Configuration · Author-reported evaluation0.631
GENA-LM (zero-shot embedding) · Configuration · Author-reported evaluation0.501
GENA-LM (probed) · Configuration · Author-reported evaluation0.466
GENA-LM (fine-tuned) · Configuration · Author-reported evaluation0.628
HyenaDNA (zero-shot likelihood) · Configuration · Author-reported evaluation0.436
HyenaDNA (zero-shot embedding) · Configuration · Author-reported evaluation0.515
HyenaDNA (probed) · Configuration · Author-reported evaluation0.467
HyenaDNA (fine-tuned) · Configuration · Author-reported evaluation0.573
Mistral-DNA (zero-shot embedding) · Configuration · Author-reported evaluation0.475
Mistral-DNA (probed) · Configuration · Author-reported evaluation0.432
Mistral-DNA (fine-tuned) · Configuration · Author-reported evaluation0.504
Nucleotide Transformer (zero-shot likelihood) · Configuration · Author-reported evaluation0.469
Nucleotide Transformer (zero-shot embedding) · Configuration · Author-reported evaluation0.613
Nucleotide Transformer (probed) · Configuration · Author-reported evaluation0.516
Nucleotide Transformer (fine-tuned) · Configuration · Author-reported evaluation0.670
ChromBPNet (ab initio) · Method · Author-reported evaluation0.892

Source order is preserved. Plotted marks show point estimates; uncertainty, where reported, is retained in the printed values and table. Differences do not establish statistical significance.

DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row group(YORUBAN), column(AUROC)

Values, uncertainty and evidence

auroc: original source values
Tested entity	Printed value	Uncertainty	Evidence
Caduceus (zero-shot likelihood) · Configuration	0.443 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban Caduceus), column(zero-shot likelihood auroc)
Caduceus (zero-shot embedding) · Configuration	0.508 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban Caduceus), column(zero-shot embedding auroc)
Caduceus (probed) · Configuration	0.490 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban Caduceus), column(probed auroc)
Caduceus (fine-tuned) · Configuration	0.666 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban Caduceus), column(fine-tuned auroc)
DNABERT-2 (zero-shot embedding) · Configuration	0.505 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban DNABERT-2), column(zero-shot embedding auroc)
DNABERT-2 (probed) · Configuration	0.476 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban DNABERT-2), column(probed auroc)
DNABERT-2 (fine-tuned) · Configuration	0.631 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban DNABERT-2), column(fine-tuned auroc)
GENA-LM (zero-shot embedding) · Configuration	0.501 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban GENA-LM), column(zero-shot embedding auroc)
GENA-LM (probed) · Configuration	0.466 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban GENA-LM), column(probed auroc)
GENA-LM (fine-tuned) · Configuration	0.628 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban GENA-LM), column(fine-tuned auroc)
HyenaDNA (zero-shot likelihood) · Configuration	0.436 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban HyenaDNA), column(zero-shot likelihood auroc)
HyenaDNA (zero-shot embedding) · Configuration	0.515 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban HyenaDNA), column(zero-shot embedding auroc)
HyenaDNA (probed) · Configuration	0.467 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban HyenaDNA), column(probed auroc)
HyenaDNA (fine-tuned) · Configuration	0.573 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban HyenaDNA), column(fine-tuned auroc)
Mistral-DNA (zero-shot embedding) · Configuration	0.475 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban Mistral-DNA), column(zero-shot embedding auroc)
Mistral-DNA (probed) · Configuration	0.432 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban Mistral-DNA), column(probed auroc)
Mistral-DNA (fine-tuned) · Configuration	0.504 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban Mistral-DNA), column(fine-tuned auroc)
Nucleotide Transformer (zero-shot likelihood) · Configuration	0.469 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban NT), column(zero-shot likelihood auroc)
Nucleotide Transformer (zero-shot embedding) · Configuration	0.613 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban NT), column(zero-shot embedding auroc)
Nucleotide Transformer (probed) · Configuration	0.516 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban NT), column(probed auroc)
Nucleotide Transformer (fine-tuned) · Configuration	0.670 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban NT), column(fine-tuned auroc)
ChromBPNet (ab initio) · Method	0.892 fraction	Not reported	Author-reported evaluation · source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban ChromBPNet), column(ab initio auroc)

Scope and limitations

Author-reported numbers, source checked but not independently reproduced.
The evaluation setting is part of the method name: a zero-shot, probed and fine-tuned run of the same model are different entries.
Metrics and datasets differ between tasks, so these figures cannot be averaged into one score.

Source transcription and grouping reviewed by automated source review on 2026-09-18. These experiments were not independently reproduced by rewire.

Tested entities and results

Release 2026-09-17-134cd1815de8 · 22 evaluations · 22 metric rows. Different protocols are not a single leaderboard. Where several source tables report the same metric, the published comparisons above offer a pooled view that names what it does not hold constant.

Evaluation setupMetricEvidence origin

Results grouped by the exact reported evaluation
Metric and finding	Coverage and uncertainty	Evidence
Caduceus (fine-tuned) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: Caduceus (fine-tuned)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.666 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban Caduceus), column(fine-tuned auroc) Source checking is not independent reproduction.
Caduceus (probed) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: Caduceus (probed)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.490 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban Caduceus), column(probed auroc) Source checking is not independent reproduction.
Caduceus (zero-shot embedding) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: Caduceus (zero-shot embedding)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.508 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban Caduceus), column(zero-shot embedding auroc) Source checking is not independent reproduction.
Caduceus (zero-shot likelihood) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: Caduceus (zero-shot likelihood)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.443 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban Caduceus), column(zero-shot likelihood auroc) Source checking is not independent reproduction.
ChromBPNet (ab initio) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Method: ChromBPNet (ab initio)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.892 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban ChromBPNet), column(ab initio auroc) Source checking is not independent reproduction.
DNABERT-2 (fine-tuned) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: DNABERT-2 (fine-tuned)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.631 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban DNABERT-2), column(fine-tuned auroc) Source checking is not independent reproduction.
DNABERT-2 (probed) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: DNABERT-2 (probed)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.476 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban DNABERT-2), column(probed auroc) Source checking is not independent reproduction.
DNABERT-2 (zero-shot embedding) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: DNABERT-2 (zero-shot embedding)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.505 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban DNABERT-2), column(zero-shot embedding auroc) Source checking is not independent reproduction.
GENA-LM (fine-tuned) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: GENA-LM (fine-tuned)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.628 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban GENA-LM), column(fine-tuned auroc) Source checking is not independent reproduction.
GENA-LM (probed) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: GENA-LM (probed)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.466 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban GENA-LM), column(probed auroc) Source checking is not independent reproduction.
GENA-LM (zero-shot embedding) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: GENA-LM (zero-shot embedding)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.501 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban GENA-LM), column(zero-shot embedding auroc) Source checking is not independent reproduction.
HyenaDNA (fine-tuned) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: HyenaDNA (fine-tuned)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.573 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban HyenaDNA), column(fine-tuned auroc) Source checking is not independent reproduction.
HyenaDNA (probed) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: HyenaDNA (probed)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.467 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban HyenaDNA), column(probed auroc) Source checking is not independent reproduction.
HyenaDNA (zero-shot embedding) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: HyenaDNA (zero-shot embedding)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.515 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban HyenaDNA), column(zero-shot embedding auroc) Source checking is not independent reproduction.
HyenaDNA (zero-shot likelihood) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: HyenaDNA (zero-shot likelihood)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.436 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban HyenaDNA), column(zero-shot likelihood auroc) Source checking is not independent reproduction.
Mistral-DNA (fine-tuned) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: Mistral-DNA (fine-tuned)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.504 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban Mistral-DNA), column(fine-tuned auroc) Source checking is not independent reproduction.
Mistral-DNA (probed) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: Mistral-DNA (probed)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.432 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban Mistral-DNA), column(probed auroc) Source checking is not independent reproduction.
Mistral-DNA (zero-shot embedding) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: Mistral-DNA (zero-shot embedding)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.475 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban Mistral-DNA), column(zero-shot embedding auroc) Source checking is not independent reproduction.
Nucleotide Transformer (fine-tuned) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: Nucleotide Transformer (fine-tuned)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.670 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban NT), column(fine-tuned auroc) Source checking is not independent reproduction.
Nucleotide Transformer (probed) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: Nucleotide Transformer (probed)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.516 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban NT), column(probed auroc) Source checking is not independent reproduction.
Nucleotide Transformer (zero-shot embedding) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: Nucleotide Transformer (zero-shot embedding)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.613 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban NT), column(zero-shot embedding auroc) Source checking is not independent reproduction.
Nucleotide Transformer (zero-shot likelihood) on DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC Configuration: Nucleotide Transformer (zero-shot likelihood)Task: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROCDataset subset: DNase QTLs in Yoruban LCLs (DART-Eval split) Score the effect of a variant on chromatin accessibility, against the measured QTL call. Author-reported evaluation · Evaluation metadata: source checked
0.469 auroc Unit: fraction · Direction: higher	Uncertainty: Not reported Scored: Not reported · Eligible: Not reported	source checkedDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Table 6, row(Yoruban NT), column(zero-shot likelihood auroc) Source checking is not independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

1 evidence row matching the loaded filters

Claims, original sources and review scope · Release 2026-09-17-134cd1815de8
Property and statement	Original source and location	Review and provenance
Relationship: part of discovery-benchmark-dart-eval Individual claims	DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA Original source ↗ Table 6, row group(YORUBAN), column(AUROC) Version: 2412.05430v1 Retrieved: 2026-09-17T07:56:09.182117+00:00	source checked automated source review · 2026-09-18 Audit details Primary-source transcription with no human sign-off and no independent reproduction. Field: `links:part_of:discovery-benchmark-dart-eval` Claim: dart-eval-association-vs-yoruban-auroc Source artifact SHA-256: `4194b137ba55c9a2c269d119a9afec6ae1bb0feaf17d91433ae483c41221a56b` Hash scope: Exact retrieved primary paper artifact bytes. Inspected artifact

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

Release 2026-09-17-134cd1815de8 · Record review: source checked

1 source records and release history

DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA · Original source · 2412.05430v1

Download this release

Technical metadata and extraction receipts

Stable ID: dart-eval-task-vs-yoruban-auroc

areas: dna-genomes
tasks: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC
metric: AUROC
metric direction: higher
dataset: DNase QTLs in Yoruban LCLs
protocol: Score the effect of a variant on chromatin accessibility, against the measured QTL call.
source locator: Table 6, row group(YORUBAN), column(AUROC)
comparison panels: id: dart-eval-panel-vs-yoruban-auroc; title: DART-Eval VS-YORUBAN-AUROC: Variant scoring on DNase QTLs in Yoruban LCLs, AUROC; protocol id: dart-eval-task-vs-yoruban-auroc; dataset id: dart-eval-dataset-dnase-qtls-in-yoruban-lcls; metric: auroc; unit: fraction; direction: higher; result ids: dart-eval-result-caduceus-zero-shot-likelihood-vs-yoruban-auroc-auroc; dart-eval-result-caduceus-zero-shot-embedding-vs-yoruban-auroc-auroc; dart-eval-result-caduceus-probed-vs-yoruban-auroc-auroc; dart-eval-result-caduceus-fine-tuned-vs-yoruban-auroc-auroc; dart-eval-result-dnabert-2-zero-shot-embedding-vs-yoruban-auroc-auroc; dart-eval-result-dnabert-2-probed-vs-yoruban-auroc-auroc; dart-eval-result-dnabert-2-fine-tuned-vs-yoruban-auroc-auroc; dart-eval-result-gena-lm-zero-shot-embedding-vs-yoruban-auroc-auroc; dart-eval-result-gena-lm-probed-vs-yoruban-auroc-auroc; dart-eval-result-gena-lm-fine-tuned-vs-yoruban-auroc-auroc; dart-eval-result-hyenadna-zero-shot-likelihood-vs-yoruban-auroc-auroc; dart-eval-result-hyenadna-zero-shot-embedding-vs-yoruban-auroc-auroc; dart-eval-result-hyenadna-probed-vs-yoruban-auroc-auroc; dart-eval-result-hyenadna-fine-tuned-vs-yoruban-auroc-auroc; dart-eval-result-mistral-dna-zero-shot-embedding-vs-yoruban-auroc-auroc; dart-eval-result-mistral-dna-probed-vs-yoruban-auroc-auroc; dart-eval-result-mistral-dna-fine-tuned-vs-yoruban-auroc-auroc; dart-eval-result-nucleotide-transformer-zero-shot-likelihood-vs-yoruban-auroc-auroc; dart-eval-result-nucleotide-transformer-zero-shot-embedding-vs-yoruban-auroc-auroc; dart-eval-result-nucleotide-transformer-probed-vs-yoruban-auroc-auroc; dart-eval-result-nucleotide-transformer-fine-tuned-vs-yoruban-auroc-auroc; dart-eval-result-chrombpnet-ab-initio-vs-yoruban-auroc-auroc; source ids: evidence-expansion-p2-evidence-discovery-final-dart-4194b137ba55; source locator: Table 6, row group(YORUBAN), column(AUROC); context: Every method DART-Eval reports on Variant scoring on DNase QTLs in Yoruban LCLs, AUROC, scored with AUROC on DNase QTLs in Yoruban LCLs.; caveats: Author-reported numbers, source checked but not independently reproduced.; The evaluation setting is part of the method name: a zero-shot, probed and fine-tuned run of the same model are different entries.; Metrics and datasets differ between tasks, so these figures cannot be averaged into one score.; review: method: automated_source_review; date: 2026-09-18

Related records

Suggest a correction