benchmark · task

MFASS splice-variant prioritisation

MFASS prioritisation tests whether model scores enrich for experimentally disrupted exon recognition.

At a glance

Explanatory profile: source reviewed · Automated source review, 2026-09-16. This does not change the review status of its results.

Data, procedure and scoring
Property	Description and evidence
Record type	Task linked to a concrete protocolMFASS v2 pinned rewire artifacts · benchmarks/mfass/README.md at bee9133b83f3aedaf2bbb9013f1875515845607e: Correction; Dataset; Cohort reconciliation; Split; results JSON in benchmarks/mfass/results/
Inputs	Functional reporter labels and variant scoresMFASS v2 pinned rewire artifacts · benchmarks/mfass/README.md at bee9133b83f3aedaf2bbb9013f1875515845607e: Correction; Dataset; Cohort reconciliation; Split; results JSON in benchmarks/mfass/results/
Assessment	Top-ranked assay positives and ranking metricsMFASS v2 pinned rewire artifacts · benchmarks/mfass/README.md at bee9133b83f3aedaf2bbb9013f1875515845607e: Correction; Dataset; Cohort reconciliation; Split; results JSON in benchmarks/mfass/results/
Datasets	Not extracted or verified for this record.
Organisms	Not extracted or verified for this record.
Assays	Not extracted or verified for this record.
Splits	Not extracted or verified for this record.
Adaptation	Not extracted or verified for this record.
Baselines	Not extracted or verified for this record.

How it works

Proposed evaluation design

Conceptual design only. This generic task has no fixed executable protocol and does not imply a completed run.

Read the diagram as text

Choose matched biological data
Define held-out evaluation
Apply candidate methods
Assess the specified endpoint

MFASS v2 pinned rewire artifacts · benchmarks/mfass/README.md at bee9133b83f3aedaf2bbb9013f1875515845607e: Correction; Dataset; Cohort reconciliation; Split; results JSON in benchmarks/mfass/results/

Procedure

The concrete rewire v2 protocol validates assay-oriented sequence pairs and evaluates a predeclared grouped split. This catalogue task describes the capability; it is distinct from the assay dataset and the pinned v2 implementation.

Connected tasks and protocols

MFASS v1 (superseded) · protocol · evaluates task
MFASS v2 · protocol · evaluates task

These associations do not imply identical protocols or interchangeable scores.

Tested models and results

Release 2026-09-16-d74d282221a9 · 4 evaluations · 12 metric rows. Different protocols are not a single leaderboard.

Evaluated configurationMetricEvidence origin

Results grouped by the exact reported evaluation
Metric and finding	Coverage and uncertainty	Evidence
Corrected k-mer / position baseline on MFASS v2 Model: Corrected k-mer / position baseline · Benchmark: MFASS v2 · Dataset: MFASS v2 eligible assay cohort Assay-oriented 21 bp k-mer window, exon position, allele identity and conservation features; gradient-boosted trees trained on the MFASS training split. Rewire evaluation · Evaluation metadata: reproduced
0.7779498064677238 auroc Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/baseline-kmer-position-v2.json :: auroc Source checking is not independent reproduction.
0.28641674595892375 average_precision Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/baseline-kmer-position-v2.json :: average_precision Source checking is not independent reproduction.
0.61 precision_at_100 Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/baseline-kmer-position-v2.json :: precision_at_100 Source checking is not independent reproduction.
DNABERT-2 117M · frozen pair embeddings on MFASS v2 Model: DNABERT-2 117M · frozen pair embeddings · Benchmark: MFASS v2 · Dataset: MFASS v2 eligible assay cohort Masked mean of frozen last hidden states for 170 bp reference and mutant sequences; concatenate reference and mutant-minus-reference embeddings; fixed balanced L2 logistic head trained only on the MFASS training split. Rewire evaluation · Evaluation metadata: reproduced
0.5500324040216661 auroc Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/dnabert2-117m-frozen-pair-logreg.json :: auroc Source checking is not independent reproduction.
0.04508654312652131 average_precision Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/dnabert2-117m-frozen-pair-logreg.json :: average_precision Source checking is not independent reproduction.
0.03 precision_at_100 Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/dnabert2-117m-frozen-pair-logreg.json :: precision_at_100 Source checking is not independent reproduction.
Pangolin · mask=False on MFASS v2 Model: Pangolin · mask=False · Benchmark: MFASS v2 · Dataset: MFASS v2 eligible assay cohort Unchanged specialist run in genomic context with GENCODE v44; zero-shot on MFASS assay labels. Point metrics use the scored subset. Rewire evaluation · Evaluation metadata: reproduced
0.8756851300560864 auroc Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8301/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/pangolin-maskFalse.json :: auroc Source checking is not independent reproduction.
0.3887617543064248 average_precision Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8301/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/pangolin-maskFalse.json :: average_precision Source checking is not independent reproduction.
0.65 precision_at_100 Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8301/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/pangolin-maskFalse.json :: precision_at_100 Source checking is not independent reproduction.
SpliceAI 1.3.1 on MFASS v2 Model: SpliceAI 1.3.1 · Benchmark: MFASS v2 · Dataset: MFASS v2 eligible assay cohort Unchanged specialist run in genomic context with bundled annotation; zero-shot on MFASS assay labels. Point metrics use the scored subset. Rewire evaluation · Evaluation metadata: reproduced
0.8055241740253153 auroc Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8194/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/spliceai-1.3.1.json :: auroc Source checking is not independent reproduction.
0.2986855472760137 average_precision Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8194/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/spliceai-1.3.1.json :: average_precision Source checking is not independent reproduction.
0.64 precision_at_100 Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8194/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/spliceai-1.3.1.json :: precision_at_100 Source checking is not independent reproduction.

Strengths and limitations

Strengths supported by sources

Experimental reporter labels offer a functional endpoint independent of clinical assertions.MFASS v2 pinned rewire artifacts · benchmarks/mfass/README.md at bee9133b83f3aedaf2bbb9013f1875515845607e: Correction; Dataset; Cohort reconciliation; Split; results JSON in benchmarks/mfass/results/

Limitations and conditions

Reporter disruption is not a clinical diagnosis, and models with different sequence context are not identical-input baselines.MFASS v2 pinned rewire artifacts · benchmarks/mfass/README.md at bee9133b83f3aedaf2bbb9013f1875515845607e: Correction; Dataset; Cohort reconciliation; Split; results JSON in benchmarks/mfass/results/

Profile review details

Primary-source description checked by an automated research assistant. This is profile review, not independent execution or numerical reproduction.

Stable record: catalog-task-mfass-splice

Sources and history

Release 2026-09-16-d74d282221a9 · Record review: discovered

DNABERT-2 official resource · Original source · Not reported
Nucleotide Transformer v2 official resource · Original source · Not reported
SpliceAI official resource · Original source · Not reported
Pangolin official resource · Original source · Not reported
MFASS v2 pinned rewire artifacts · Original source · bee9133b83f3aedaf2bbb9013f1875515845607e

Download this release

Technical metadata and extraction receipts

Stable ID: catalog-task-mfass-splice

areas: dna-genomes
entity level: task
version: Not reported
task: MFASS splice-variant prioritisation
scope note: Functional exon-recognition assay; mfass-v2 reports a corrected baseline and one complete local DNABERT-2 protocol.
missing metadata: protocol version: not_yet_extracted

Related records

evaluates task: MFASS v1 (superseded)
evaluates task: MFASS v2

Suggest a correction