Evaluation

Training class prior / majority on MFASS v2 canonical held-out split

Constant training-class prior scored on all 8,324 canonical MFASS v2 held-out variants. Tied top-100 predictions are one fixed label-independent ordering, not ranking ability.

Evaluation results

1 evaluation · 3 metric rows. Different protocols are not a single leaderboard.

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: Training class prior / majority (MFASS v2 canonical held-out split)	Protocol: MFASS v2 Dataset: MFASS v2 eligible assay cohort	0.5 auroc dimensionless · higher Uncertainty: Not reported Coverage: 8324/8324	Rewire evaluation · source checked Methods, coverage and source Training class prior / majority on MFASS v2 canonical held-out split MFASS v2 canonical held-out split Aggregation: Not reported Training class prior / majority: report; Training class prior / majority: audit · metrics.auroc
Configuration: Training class prior / majority (MFASS v2 canonical held-out split)	Protocol: MFASS v2 Dataset: MFASS v2 eligible assay cohort	0.037842383469485825 average_precision_sklearn dimensionless · higher Uncertainty: Not reported Coverage: 8324/8324	Rewire evaluation · source checked Methods, coverage and source Training class prior / majority on MFASS v2 canonical held-out split MFASS v2 canonical held-out split Aggregation: Not reported Training class prior / majority: report; Training class prior / majority: audit · metrics.average_precision_sklearn
Configuration: Training class prior / majority (MFASS v2 canonical held-out split)	Protocol: MFASS v2 Dataset: MFASS v2 eligible assay cohort	0.04 precision_at_capacity dimensionless · higher Uncertainty: Not reported Coverage: 8324/8324	Rewire evaluation · source checked Methods, coverage and source Training class prior / majority on MFASS v2 canonical held-out split MFASS v2 canonical held-out split Aggregation: Not reported Training class prior / majority: report; Training class prior / majority: audit · metrics.precision_at_capacity

Source checking is not independent reproduction. Release 2026-09-22-f58a0f1d267f.

Evaluation procedure

MFASS v2 canonical held-out split

Configuration: Training class prior / majority (MFASS v2 canonical held-out split)
Protocol: MFASS v2
Dataset: MFASS v2 eligible assay cohort

origin: Rewire evaluation
configuration: bee9133b83f3aedaf2bbb9013f1875515845607e
protocol id: rewire-mfass-v2
dataset version: rewire-mfass-v2-dataset
split: split-v2.tsv
subset: mfass-assay-pairs-v2
population: 8324/8324
inputs: protocol allowlisted biological inputs; see baseline configuration for extra inputs
adaptation: training class prior; no sequence fitting
metric implementation: 08cbbaac2aa8f60af5c43af7a83410e9690b44c5e4d162fbe1e5766885952fce
aggregation: all held-out test rows of the selected target
budget: one local CPU evaluation; no held-out hyperparameter selection

Metadata review: source checked. Unreported conditions prevent automatic comparisons.

Reproduction

Split: split-v2.tsv
Adaptation: training class prior; no sequence fitting
Scoring implementation: 08cbbaac2aa8f60af5c43af7a83410e9690b44c5e4d162fbe1e5766885952fce

No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.

Access requirements and official instructions: MFASS v2

Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

216 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-22-f58a0f1d267f
Property and statement	Original source and location	Review and provenance
attributes.comparison.adaptation training class prior; no sequence fitting Context-only references	Training class prior / majority: audit Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.adaptation` Source artifact SHA-256: `7eff5c6fe57c641296be48cce0ac90ab9c03f59db2c1fc43c5fafd41a93c76c8` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.adaptation training class prior; no sequence fitting Context-only references	Training class prior / majority: execution and verification instructions Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.adaptation` Source artifact SHA-256: `2115525b87c4a4f44b0c7f3e6ff41aca3aa2b2cd5ffe0590c633e210bb559fe5` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.adaptation training class prior; no sequence fitting Context-only references	Training class prior / majority: report Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.adaptation` Source artifact SHA-256: `89e06a2f7229f16b980254a6589eb808ec89137018388db98719beb72324f56a` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.aggregation all held-out test rows of the selected target Context-only references	Training class prior / majority: audit Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.aggregation` Source artifact SHA-256: `7eff5c6fe57c641296be48cce0ac90ab9c03f59db2c1fc43c5fafd41a93c76c8` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.aggregation all held-out test rows of the selected target Context-only references	Training class prior / majority: execution and verification instructions Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.aggregation` Source artifact SHA-256: `2115525b87c4a4f44b0c7f3e6ff41aca3aa2b2cd5ffe0590c633e210bb559fe5` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.aggregation all held-out test rows of the selected target Context-only references	Training class prior / majority: report Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.aggregation` Source artifact SHA-256: `89e06a2f7229f16b980254a6589eb808ec89137018388db98719beb72324f56a` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.budget one local CPU evaluation; no held-out hyperparameter selection Context-only references	Training class prior / majority: audit Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.budget` Source artifact SHA-256: `7eff5c6fe57c641296be48cce0ac90ab9c03f59db2c1fc43c5fafd41a93c76c8` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.budget one local CPU evaluation; no held-out hyperparameter selection Context-only references	Training class prior / majority: execution and verification instructions Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.budget` Source artifact SHA-256: `2115525b87c4a4f44b0c7f3e6ff41aca3aa2b2cd5ffe0590c633e210bb559fe5` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.budget one local CPU evaluation; no held-out hyperparameter selection Context-only references	Training class prior / majority: report Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.budget` Source artifact SHA-256: `89e06a2f7229f16b980254a6589eb808ec89137018388db98719beb72324f56a` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.dataset_version rewire-mfass-v2-dataset Context-only references	Training class prior / majority: audit Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.dataset_version` Source artifact SHA-256: `7eff5c6fe57c641296be48cce0ac90ab9c03f59db2c1fc43c5fafd41a93c76c8` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-22-f58a0f1d267f · Record review: source checked

3 source records and release history

Training class prior / majority: report · Original source · 1663d1f04b2bbd6dfcff77fea78129d30b0de191
Training class prior / majority: audit · Original source · 1663d1f04b2bbd6dfcff77fea78129d30b0de191
Training class prior / majority: execution and verification instructions · Original source · 1663d1f04b2bbd6dfcff77fea78129d30b0de191

Download this release

Technical metadata and extraction receipts

Stable ID: rewire-local-20260921-evaluation-mfass-prior

areas: dna-genomes
origin: rewire_run
execution scope: complete_selected_evaluation
published score reproduction: false
protocol: MFASS v2 canonical held-out split
version: bee9133b83f3aedaf2bbb9013f1875515845607e
source locator: metrics; coverage; protocol_results
original sdk scope: full
original sdk completion: complete
suite complete: false
eligible count: 8324
scored count: 8324
missing count: 0
coverage: 8324/8324
comparison: protocol id: rewire-mfass-v2; dataset version: rewire-mfass-v2-dataset; split: split-v2.tsv; subset: mfass-assay-pairs-v2; population: 8324/8324; inputs: protocol allowlisted biological inputs; see baseline configuration for extra inputs; adaptation: training class prior; no sequence fitting; metric implementation: 08cbbaac2aa8f60af5c43af7a83410e9690b44c5e4d162fbe1e5766885952fce; aggregation: all held-out test rows of the selected target; budget: one local CPU evaluation; no held-out hyperparameter selection
input information: protocol allowlisted biological inputs; see baseline configuration for extra inputs
model configuration: baseline id: training-prior-v1; baseline version: 1; binary: positive-class prevalence; multiclass: majority; lowest-class tie
protocol configuration: canonical test count: 8324; canonical train count: 19409; capacity: 100; head: C: 0.1; class weight: balanced; max iter: 1000; seed: 20260914; solver: liblinear; input context: 170bp assay-oriented pair; smoke limit per split: Not reported; source reuse terms: unreported; source data not redistributed
provenance: annotation sha256: 71a857fe647c4e68acbb41ca61e959c47e1176de89b1442bd6ca1772aa60d5a1; cohort sha256: 389702ff4c647d7ce10a90092a6fa811ae777d15997baf39ce9aae0346247bd0; raw sha256: a637ca0e307e66ff48811ec7efa22b9ce453bc7883b04f0cacb867f7283132d8; runner revision: bee9133b83f3aedaf2bbb9013f1875515845607e; source url: https://github.com/rewire-bio/rewire-benchmarks/tree/bee9133b83f3aedaf2bbb9013f1875515845607e/benchmarks/mfass; split sha256: 999ebcb7e63a5c5eaa8780fa468e59ac1f934260ad50102814174c396317f052; upstream revision: 9a8e4f27106be52aeb11acad27f95f5cded663a8
predictions sha256: c9f2d729201ae3dd80fcfa7eb71c75968eb6cb33e30648109e1cbda070a3b9f8
prepared sha256: cbba55d19fea7cb204127343ecb554da169a16f047117eb1db6472a5707f38d3
execution: adapter: TrainingPrior; batch size: 32; fitting: train_only; inference and fit seconds: 0.09625304200744722; mode: local_adapter; prediction type: scalar
timing seconds: evaluation: 0.44232487499539275
environment: architecture: arm64; container digest: unreported; container runtime: unreported; platform: macOS-26.6.2-arm64-arm-64bit; python: 3.11.13; rewirebench: 0.4.0; sdk code sha256: 08cbbaac2aa8f60af5c43af7a83410e9690b44c5e4d162fbe1e5766885952fce; sif sha256: unreported
run url: https://github.com/rewire-bio/rewire-benchmarks/blob/1663d1f04b2bbd6dfcff77fea78129d30b0de191/research/mfass-null-2026-09-21/evidence/training-prior.report.json
reproduction url: https://github.com/rewire-bio/rewire-benchmarks/blob/1663d1f04b2bbd6dfcff77fea78129d30b0de191/research/mfass-null-2026-09-21/README.md
reproduction note: The linked pinned scripts describe this exact execution. No generic suite recipe is claimed to reproduce it; this review only checked existing artifacts.
limitations: No confidence intervals or seed variability estimate.; Source checking and local execution do not reproduce a published paper score.; All predictions tie. Four positives in the first 100 reflect one fixed label-independent tie break, not expected chance ranking ability.

Related records

configuration: Training class prior / majority (MFASS v2 canonical held-out split)
protocol: MFASS v2
dataset: MFASS v2 eligible assay cohort
evaluation: Training class prior / majority: auroc
evaluation: Training class prior / majority: average_precision_sklearn
evaluation: Training class prior / majority: precision_at_capacity

Suggest a correction