benchmark · protocol

MFASS v2

MFASS v2 ranks splice-disrupting variants using corrected, assay-oriented sequence inputs.

At a glance

Explanatory profile: source reviewed · Automated source review, 2026-09-16. This does not change the review status of its results.

Data, procedure and scoring
Property	Description and evidence
Record type	Pinned rewire protocolMFASS v2 pinned rewire artifacts · benchmarks/mfass/README.md at bee9133b83f3aedaf2bbb9013f1875515845607e: Correction; Dataset; Cohort reconciliation; Split; results JSON in benchmarks/mfass/results/
Inputs	MFASS functional assay labels; assay-oriented variant pairs or specialist genomic inputsMFASS v2 pinned rewire artifacts · benchmarks/mfass/README.md at bee9133b83f3aedaf2bbb9013f1875515845607e: Correction; Dataset; Cohort reconciliation; Split; results JSON in benchmarks/mfass/results/
Assessment	Variant ranking on the held-out assay setMFASS v2 pinned rewire artifacts · benchmarks/mfass/README.md at bee9133b83f3aedaf2bbb9013f1875515845607e: Correction; Dataset; Cohort reconciliation; Split; results JSON in benchmarks/mfass/results/
Eligible cohort / train / test	27,733 / 19,409 / 8,324 variantsMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/baseline-kmer-position-v2.json; README: Cohort reconciliation and Reproduce v2
Corrected sequence handling	Reference natural_seq and mutant original_seq are validated at rel_position; 7,770 eligible raw sequence fields were reverse complemented.MFASS v2 pinned rewire artifacts · README: Correction and Cohort reconciliation
Baseline	Gradient-boosted trees using a correctly centred 21 bp k-mer window, exon position, alleles and conservation.MFASS v2 pinned rewire artifacts · benchmarks/mfass/results/baseline-kmer-position-v2.json
Organisms	Not extracted or verified for this record.
Assays	Not extracted or verified for this record.
Splits	Not extracted or verified for this record.
Adaptation	Not extracted or verified for this record.

How it works

Procedure overview

Conceptual overview, not an executable specification.

Read the diagram as text

Validate assay-oriented pairs
Use pinned grouped split
Score held-out variants
Assess ranking and coverage

MFASS v2 pinned rewire artifacts · benchmarks/mfass/README.md at bee9133b83f3aedaf2bbb9013f1875515845607e: Correction; Dataset; Cohort reconciliation; Split; results JSON in benchmarks/mfass/results/

Procedure

Validate reference and mutant sequences, apply the predeclared gene-and-exon connected-component split, then rank held-out variants. Report top-100 precision, average precision and AUROC; paired comparisons use common scored variants and group-aware uncertainty.

Connected tasks and protocols

MFASS splice-variant prioritisation · task · evaluates task

These associations do not imply identical protocols or interchangeable scores.

Tested models and results

Release 2026-09-16-d74d282221a9 · 4 evaluations · 12 metric rows. Different protocols are not a single leaderboard.

Evaluated configurationMetricEvidence origin

Results grouped by the exact reported evaluation
Metric and finding	Coverage and uncertainty	Evidence
Corrected k-mer / position baseline on MFASS v2 Model: Corrected k-mer / position baseline · Benchmark: MFASS v2 · Dataset: MFASS v2 eligible assay cohort Assay-oriented 21 bp k-mer window, exon position, allele identity and conservation features; gradient-boosted trees trained on the MFASS training split. Rewire evaluation · Evaluation metadata: reproduced
0.7779498064677238 auroc Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/baseline-kmer-position-v2.json :: auroc Source checking is not independent reproduction.
0.28641674595892375 average_precision Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/baseline-kmer-position-v2.json :: average_precision Source checking is not independent reproduction.
0.61 precision_at_100 Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/baseline-kmer-position-v2.json :: precision_at_100 Source checking is not independent reproduction.
DNABERT-2 117M · frozen pair embeddings on MFASS v2 Model: DNABERT-2 117M · frozen pair embeddings · Benchmark: MFASS v2 · Dataset: MFASS v2 eligible assay cohort Masked mean of frozen last hidden states for 170 bp reference and mutant sequences; concatenate reference and mutant-minus-reference embeddings; fixed balanced L2 logistic head trained only on the MFASS training split. Rewire evaluation · Evaluation metadata: reproduced
0.5500324040216661 auroc Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/dnabert2-117m-frozen-pair-logreg.json :: auroc Source checking is not independent reproduction.
0.04508654312652131 average_precision Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/dnabert2-117m-frozen-pair-logreg.json :: average_precision Source checking is not independent reproduction.
0.03 precision_at_100 Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8324/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/dnabert2-117m-frozen-pair-logreg.json :: precision_at_100 Source checking is not independent reproduction.
Pangolin · mask=False on MFASS v2 Model: Pangolin · mask=False · Benchmark: MFASS v2 · Dataset: MFASS v2 eligible assay cohort Unchanged specialist run in genomic context with GENCODE v44; zero-shot on MFASS assay labels. Point metrics use the scored subset. Rewire evaluation · Evaluation metadata: reproduced
0.8756851300560864 auroc Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8301/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/pangolin-maskFalse.json :: auroc Source checking is not independent reproduction.
0.3887617543064248 average_precision Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8301/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/pangolin-maskFalse.json :: average_precision Source checking is not independent reproduction.
0.65 precision_at_100 Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8301/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/pangolin-maskFalse.json :: precision_at_100 Source checking is not independent reproduction.
SpliceAI 1.3.1 on MFASS v2 Model: SpliceAI 1.3.1 · Benchmark: MFASS v2 · Dataset: MFASS v2 eligible assay cohort Unchanged specialist run in genomic context with bundled annotation; zero-shot on MFASS assay labels. Point metrics use the scored subset. Rewire evaluation · Evaluation metadata: reproduced
0.8055241740253153 auroc Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8194/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/spliceai-1.3.1.json :: auroc Source checking is not independent reproduction.
0.2986855472760137 average_precision Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8194/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/spliceai-1.3.1.json :: average_precision Source checking is not independent reproduction.
0.64 precision_at_100 Unit: fraction · Direction: higher	Uncertainty: Point estimate; see paired-comparison artifacts. Coverage (scored/eligible): 8194/8324	reproducedMFASS v2 pinned rewire artifacts · benchmarks/mfass/results/spliceai-1.3.1.json :: precision_at_100 Source checking is not independent reproduction.

Strengths and limitations

Strengths supported by sources

A fixed review capacity connects ranking quality to a limited follow-up budget.MFASS v2 pinned rewire artifacts · benchmarks/mfass/README.md at bee9133b83f3aedaf2bbb9013f1875515845607e: Correction; Dataset; Cohort reconciliation; Split; results JSON in benchmarks/mfass/results/

Limitations and conditions

The reporter measures exon recognition in an artificial construct, not clinical pathogenicity. Methods have different input context and scoring coverage.MFASS v2 pinned rewire artifacts · benchmarks/mfass/README.md at bee9133b83f3aedaf2bbb9013f1875515845607e: Correction; Dataset; Cohort reconciliation; Split; results JSON in benchmarks/mfass/results/

Profile review details

Primary-source description checked by an automated research assistant. This is profile review, not independent execution or numerical reproduction.

Stable record: rewire-mfass-v2

Sources and history

Release 2026-09-16-d74d282221a9 · Record review: source checked

MFASS v2 pinned rewire artifacts · Original source · bee9133b83f3aedaf2bbb9013f1875515845607e

Supersedes MFASS v1 (superseded)

Download this release

Technical metadata and extraction receipts

Stable ID: rewire-mfass-v2

areas: dna-genomes
entity level: protocol
version: bee9133b83f3aedaf2bbb9013f1875515845607e
task: Splice-variant prioritisation
scope note: The mfass-v1 baseline used a mis-centred k-mer window for 7,770 assay variants whose raw sequence was reverse-complemented. mfass-v2 validates assay-oriented reference and mutant pairs and rebuilds the baseline; mfass-v1 remains a historical record.
missing metadata

Related records

supersedes: MFASS v1 (superseded)
evaluates task: MFASS splice-variant prioritisation
subject: MFASS v2: evaluates task MFASS splice-variant prioritisation
benchmark: Corrected k-mer / position baseline on MFASS v2
benchmark: DNABERT-2 117M · frozen pair embeddings on MFASS v2
benchmark: Pangolin · mask=False on MFASS v2
benchmark: SpliceAI 1.3.1 on MFASS v2

Suggest a correction