rewire.it
Protocol

mRNABench Sample designed MRL

Predict target_mrl_designed from sequence on the complete 15,003-record canonical test split. Train-only RidgeCV differs from upstream default validation evaluation; this is not an aggregate mRNABench score.

2 evaluations · 6 metric rows

Overview

Predict target_mrl_designed from sequence on the complete 15,003-record canonical test split. Train-only RidgeCV differs from upstream default validation evaluation; this is not an aggregate mRNABench score.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

Results

Each comparison retains its reviewed evaluation scope, dataset and metric. Results are shown without a pooled ranking.

mRNABench Sample designed MRL: local controls

mse (squared mean ribosome load) · Lower values are better.

Matched local controls on the same complete selected test split; not a comparison with published model scores.

mRNABench Sample designed MRL · mRNABench Sample designed MRL test subset

Evidence origin: Rewire evaluation. Numerical source review does not establish independent reproduction.

Sequence composition + RidgeCV: local execution report (20 September 2026); Training-mean control: local execution report (20 September 2026) · Execution reports: metrics.mse

One execution per configuration; uncertainty not estimated.

All comparison limitations (2)
  • One execution per configuration; uncertainty not estimated.
  • This selected protocol is not a benchmark-wide aggregate.

Automated source review: 2026-09-20.

No unavailable values; missing scores remain labelled and are never plotted as zero.

Showing 2 of 2 matching rows. Source order is preserved.

Tested configuration
1.892.012.142.272.39
Reported score
  1. Sequence composition + RidgeCV (mRNABench Sample designed MRL)1.914439715839851
  2. Training-mean control (mRNABench Sample designed MRL)2.3655294722554605

Dots show point estimates. Whiskers show only explicitly defined uncertainty (standard deviation, standard error or a labelled interval); their definitions remain in Table. Unresolved uncertainty is not plotted. Differences do not establish statistical significance.

Methods and evaluation design

Procedure, tasks and evaluated configurations

Evaluation design

Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.

Benchmarks

These source-backed links do not make different protocols or scores interchangeable.

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Run this benchmark

Repeat the four local sequence controls

Predict target_mrl_designed from sequence on the complete 15,003-record canonical test split. Train-only RidgeCV differs from upstream default validation evaluation; this is not an aggregate mRNABench score.

Generate predictions and evaluate them. This recipe does not establish reproduction of a particular published score.

Dataset access
Obtain the permitted source files and verify hashes following the cited instructions before running.
Model and weights
No pretrained weights; procedural controls fit only training labels.
Licences
Follow original dataset and software reuse terms. No dataset or weight redistribution is included.
Software
Pinned runner checkout; Python 3.11.13 and rewirebench 0.4.0 wheel. Install the exact executed dependency list linked in the instructions.
Hardware
Executed on macOS arm64 CPU, one thread. No accelerator required; memory and cross-platform performance unreported.
Required inputs and expected outputs

Inputs

  • Pinned Rhomax by_wild_type CSV and Sample designed parquet; script evaluates both datasets and both controls

Outputs

  • Private predictions, coverage report, metrics and sanitized contribution bundle

Execution steps

  1. 1. Run from the pinned runner checkout after preparation (Command line)

    Source reviewed; these instructions have not been executed by rewire.

    OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 python research/local-runs-2026-09-20/run_sequence.py \
  --flip-source local-inputs/by_wild_type.csv.gz \
  --mrna-source local-inputs/mrl-sample-designed.parquet \
  --work local-execution --evidence local-evidence
    mRNABench Sample designed MRL: reproduction instructions · Reproduce section; replace paths with prepared local inputs

Use your own model

Run your model locally and return predictions keyed by the input IDs. The evaluator supplies biological inputs without test labels and owns scoring. This interface is not a sandbox for model code.

Pass your existing prediction function into this adapter. Its output direction must match the selected protocol.

class MyModelAdapter:
    def __init__(self, score):
        self.score = score

    def predict(self, inputs):
        return {row["id"]: float(self.score(row)) for row in inputs}

# adapter = MyModelAdapter(your_prediction_function)
# report = rewirebench.run(prepared, adapter, output="runs/my-model")

Alternatively, generate a keyed prediction file in your existing model environment and use the score-only recipe. Your model code and weights do not need to be shared.

mRNABench Sample designed MRL: reproduction instructions · Reproduction instructions and execution scope
Scope and limitations
  • The script was executed locally, but these portable path examples have not been rerun verbatim.
  • Only the selected assay or selected sequence controls are evaluated; no suite aggregate.
  • Use new output directories. Raw predictions and private inputs are not uploaded.
  • Matching a fresh local result is distinct from reproducing an earlier paper score.

Contribute a result for review. The library can submit an exported evaluation for private review when intake is open. Check the contribution page for access and sign-in.

Original repository instructions

Run instructions

No runnable recipe has been reviewed for this protocol. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

Strengths, limitations and unresolved questions

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

7 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-22-f58a0f1d267f
Property and statementOriginal source and locationReview and provenance
Run recipe: inputs

["Pinned Rhomax by_wild_type CSV and Sample designed parquet; script evaluates both datasets and both controls"]

Individual claims		mRNABench Sample designed MRL: reproduction instructions

Original source ↗

Reproduction instructions and execution scope

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

source checked

implementation review

Audit details

Recipe metadata reviewed against pinned sources; not experimental reproduction.

Field: attributes.run_recipes.0.inputs

Source artifact SHA-256: 75f50703210dc556d77fdfe8f1d28c3200121b5b78c5c6592a096d928b4ff7cc

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Run recipe: Run from the pinned runner checkout after preparation

OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 python research/local-runs-2026-09-20/run_sequence.py \ --flip-source local-inputs/by_wild_type.csv.gz \ --mrna-source local-inputs/mrl-sample-designed.parquet \ --work local-execution --evidence local-evidence

Individual claims		mRNABench Sample designed MRL: reproduction instructions

Original source ↗

Reproduce section; replace paths with prepared local inputs

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

source checked

implementation review

Audit details

Execution status: source_reviewed_not_executed. Execution receipts apply only to their stated scope; not experimental reproduction.

Field: attributes.run_recipes.0.instructions.0.code

Source artifact SHA-256: 75f50703210dc556d77fdfe8f1d28c3200121b5b78c5c6592a096d928b4ff7cc

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Run recipe: limitations

["The script was executed locally, but these portable path examples have not been rerun verbatim.","Only the selected assay or selected sequence controls are evaluated; no suite aggregate.","Use new output directories. Raw predictions and private inputs are not uploaded.","Matching a fresh local result is distinct from reproducing an earlier paper score."]

Individual claims		mRNABench Sample designed MRL: reproduction instructions

Original source ↗

Reproduction instructions and execution scope

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

source checked

implementation review

Audit details

Recipe metadata reviewed against pinned sources; not experimental reproduction.

Field: attributes.run_recipes.0.limitations

Source artifact SHA-256: 75f50703210dc556d77fdfe8f1d28c3200121b5b78c5c6592a096d928b4ff7cc

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Run recipe: outputs

["Private predictions, coverage report, metrics and sanitized contribution bundle"]

Individual claims		mRNABench Sample designed MRL: reproduction instructions

Original source ↗

Reproduction instructions and execution scope

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

source checked

implementation review

Audit details

Recipe metadata reviewed against pinned sources; not experimental reproduction.

Field: attributes.run_recipes.0.outputs

Source artifact SHA-256: 75f50703210dc556d77fdfe8f1d28c3200121b5b78c5c6592a096d928b4ff7cc

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Run recipe: requirements

{"data":"Obtain the permitted source files and verify hashes following the cited instructions before running.","weights":"No pretrained weights; procedural controls fit only training labels.","licence":"Follow original dataset and software reuse terms. No dataset or weight redistribution is included.","software":"Pinned runner checkout; Python 3.11.13 and rewirebench 0.4.0 wheel. Install the exact executed dependency list linked in the instructions.","hardware":"Executed on macOS arm64 CPU, one thread. No accelerator required; memory and cross-platform performance unreported."}

Individual claims		mRNABench Sample designed MRL: reproduction instructions

Original source ↗

Reproduction instructions and execution scope

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

source checked

implementation review

Audit details

Recipe metadata reviewed against pinned sources; not experimental reproduction.

Field: attributes.run_recipes.0.requirements

Source artifact SHA-256: 75f50703210dc556d77fdfe8f1d28c3200121b5b78c5c6592a096d928b4ff7cc

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Run recipe: summary

Predict target_mrl_designed from sequence on the complete 15,003-record canonical test split. Train-only RidgeCV differs from upstream default validation evaluation; this is not an aggregate mRNABench score.

Individual claims		mRNABench Sample designed MRL: reproduction instructions

Original source ↗

Reproduction instructions and execution scope

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

source checked

implementation review

Audit details

Recipe metadata reviewed against pinned sources; not experimental reproduction.

Field: attributes.run_recipes.0.summary

Source artifact SHA-256: 75f50703210dc556d77fdfe8f1d28c3200121b5b78c5c6592a096d928b4ff7cc

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Relationship: part of

discovery-benchmark-mrnabench

Individual claims		Sequence composition + RidgeCV: local execution report (20 September 2026)

Original source ↗

protocol_id; protocol_configuration; provenance

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

source checked

automated execution evidence review · 2026-09-20T20:54:03Z

Audit details

Field: links:part_of:discovery-benchmark-mrnabench

Claim: rewire-protocol-mrnabench-designed-mrl-v1-part-of-discovery-benchmark-mrnabench

Source artifact SHA-256: 8b68dfe5725bf5d8c108265039e3787f4b3b55986eab8a430082c4d446a4f960

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-22-f58a0f1d267f · Record review: source checked

2 source records and release historyDownload this release
Technical metadata and extraction receipts

Stable ID: rewire-protocol-mrnabench-designed-mrl-v1

areas
rna-transcriptomics
protocol id
mrnabench-sample-mrl-v1
version
1
procedure
adapter input contract: opaque-sequence-only-v1; canonical test count: 15003; dataset: designed; eligible count: 100017; excluded: None recorded; head: alphas: 0.001; 0.01; 0.1; 1; 10; fit split: train; normalization: none; type: RidgeCV; input context: Full processed sequence without CDS/splice/derived tracks; metric: mse; metric direction: lower; scope note: One Sample dataset/target only, not a complete mRNABench score; seed: 2541; smoke limit per split: Not reported; source count: 100017; split counts: test: 15003; train: 70011; validation: 15003; split ratios: 0.7; 0.15; 0.15; target: target_mrl_designed; task: regression
source locator
protocol_configuration; protocol_results; provenance
access note
See the pinned execution report and source retrieval evidence. Dataset reuse terms are source-specific.
reproducibility note
Fresh local evaluation; reference metric agreement is not reproduction of a paper score.
reproduction url
https://github.com/rewire-bio/rewire-benchmarks/blob/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/README.md
run recipes
id: local-20260920-mrnabench-repeat; protocol id: rewire-protocol-mrnabench-designed-mrl-v1; version: ca73fa47136d182f2d4ddb083d084712198fc0e2; title: Repeat the four local sequence controls; purpose: generate_and_evaluate; summary: Predict target_mrl_designed from sequence on the complete 15,003-record canonical test split. Train-only RidgeCV differs from upstream default validation evaluation; this is not an aggregate mRNABench score.; inputs: Pinned Rhomax by_wild_type CSV and Sample designed parquet; script evaluates both datasets and both controls; outputs: Private predictions, coverage report, metrics and sanitized contribution bundle; requirements: data: Obtain the permitted source files and verify hashes following the cited instructions before running.; weights: No pretrained weights; procedural controls fit only training labels.; licence: Follow original dataset and software reuse terms. No dataset or weight redistribution is included.; software: Pinned runner checkout; Python 3.11.13 and rewirebench 0.4.0 wheel. Install the exact executed dependency list linked in the instructions.; hardware: Executed on macOS arm64 CPU, one thread. No accelerator required; memory and cross-platform performance unreported.; instructions: runtime: command_line; title: Run from the pinned runner checkout after preparation; code: OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 python research/local-runs-2026-09-20/run_sequence.py \ --flip-source local-inputs/by_wild_type.csv.gz \ --mrna-source local-inputs/mrl-sample-designed.parquet \ --work local-execution --evidence local-evidence; status: source_reviewed_not_executed; source ids: rewire-local-20260920-instructions-mrnabench; source locator: Reproduce section; replace paths with prepared local inputs; limitations: The script was executed locally, but these portable path examples have not been rerun verbatim.; Only the selected assay or selected sequence controls are evaluated; no suite aggregate.; Use new output directories. Raw predictions and private inputs are not uploaded.; Matching a fresh local result is distinct from reproducing an earlier paper score.; source ids: rewire-local-20260920-instructions-mrnabench; source locator: Reproduction instructions and execution scope
comparison panels
id: local-20260920-mrnabench-mse; title: mRNABench Sample designed MRL: local controls; protocol id: rewire-protocol-mrnabench-designed-mrl-v1; dataset id: rewire-dataset-mrnabench-designed-mrl-v1; metric: mse; unit: squared mean ribosome load; direction: lower; result ids: rewire-local-20260920-result-mrnabench-composition-mse; rewire-local-20260920-result-mrnabench-train-mean-mse; source ids: rewire-local-20260920-source-mrnabench-composition; rewire-local-20260920-source-mrnabench-train-mean; source locator: Execution reports: metrics.mse; context: Matched local controls on the same complete selected test split; not a comparison with published model scores.; caveats: One execution per configuration; uncertainty not estimated.; This selected protocol is not a benchmark-wide aggregate.; review: method: automated_source_review; date: 2026-09-20
Related records

Suggest a correction