MFASS v2 eligible assay cohort
Evaluation results
Release 2026-09-16-d74d282221a9 · 4 evaluations · 12 metric rows. Different protocols are not a single leaderboard.
| Metric and finding | Coverage and uncertainty | Evidence |
|---|---|---|
| Corrected k-mer / position baseline on MFASS v2 Model: Corrected k-mer / position baseline · Benchmark: MFASS v2 · Dataset: MFASS v2 eligible assay cohort Assay-oriented 21 bp k-mer window, exon position, allele identity and conservation features; gradient-boosted trees trained on the MFASS training split. Rewire evaluation · Evaluation metadata: reproduced | ||
| 0.7779498064677238 auroc Unit: fraction · Direction: higher | Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324 | reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/baseline-kmer-position-v2.json :: auroc Source checking is not independent reproduction. |
| 0.28641674595892375 average_precision Unit: fraction · Direction: higher | Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324 | reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/baseline-kmer-position-v2.json :: average_precision Source checking is not independent reproduction. |
| 0.61 precision_at_100 Unit: fraction · Direction: higher | Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324 | reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/baseline-kmer-position-v2.json :: precision_at_100 Source checking is not independent reproduction. |
| DNABERT-2 117M · frozen pair embeddings on MFASS v2 Model: DNABERT-2 117M · frozen pair embeddings · Benchmark: MFASS v2 · Dataset: MFASS v2 eligible assay cohort Masked mean of frozen last hidden states for 170 bp reference and mutant sequences; concatenate reference and mutant-minus-reference embeddings; fixed balanced L2 logistic head trained only on the MFASS training split. Rewire evaluation · Evaluation metadata: reproduced | ||
| 0.5500324040216661 auroc Unit: fraction · Direction: higher | Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324 | reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/dnabert2-117m-frozen-pair-logreg.json :: auroc Source checking is not independent reproduction. |
| 0.04508654312652131 average_precision Unit: fraction · Direction: higher | Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324 | reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/dnabert2-117m-frozen-pair-logreg.json :: average_precision Source checking is not independent reproduction. |
| 0.03 precision_at_100 Unit: fraction · Direction: higher | Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324 | reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/dnabert2-117m-frozen-pair-logreg.json :: precision_at_100 Source checking is not independent reproduction. |
| Pangolin · mask=False on MFASS v2 Unchanged specialist run in genomic context with GENCODE v44; zero-shot on MFASS assay labels. Point metrics use the scored subset. Rewire evaluation · Evaluation metadata: reproduced | ||
| 0.8756851300560864 auroc Unit: fraction · Direction: higher | Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8301/8324 | reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/pangolin-maskFalse.json :: auroc Source checking is not independent reproduction. |
| 0.3887617543064248 average_precision Unit: fraction · Direction: higher | Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8301/8324 | reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/pangolin-maskFalse.json :: average_precision Source checking is not independent reproduction. |
| 0.65 precision_at_100 Unit: fraction · Direction: higher | Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8301/8324 | reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/pangolin-maskFalse.json :: precision_at_100 Source checking is not independent reproduction. |
| SpliceAI 1.3.1 on MFASS v2 Unchanged specialist run in genomic context with bundled annotation; zero-shot on MFASS assay labels. Point metrics use the scored subset. Rewire evaluation · Evaluation metadata: reproduced | ||
| 0.8055241740253153 auroc Unit: fraction · Direction: higher | Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8194/8324 | reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/spliceai-1.3.1.json :: auroc Source checking is not independent reproduction. |
| 0.2986855472760137 average_precision Unit: fraction · Direction: higher | Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8194/8324 | reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/spliceai-1.3.1.json :: average_precision Source checking is not independent reproduction. |
| 0.64 precision_at_100 Unit: fraction · Direction: higher | Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8194/8324 | reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/spliceai-1.3.1.json :: precision_at_100 Source checking is not independent reproduction. |
Sources and history
Release 2026-09-16-d74d282221a9 · Record review: source checked
- MFASS v2 pinned rewire artifacts · Original source · bee9133b83f3aedaf2bbb9013f1875515845607e
Technical metadata and extraction receipts
Stable ID: rewire-mfass-v2-dataset
- areas
- dna-genomes
- version
- bee9133b83f3aedaf2bbb9013f1875515845607e
- split
- split-v2.tsv
- cohort variants
- 27733
- train variants
- 19409
- test variants
- 8324
- test positives
- 315
- independent test groups
- 463
- missing metadata