rewire.it
Protocol

FLIP2 Rhomax by_wild_type

Predict measured Rhomax wavelengths (nm) from amino-acid sequence on all 184 held-out archived test records. This selected split is not a complete FLIP2 score.

5 evaluations · 10 metric rows

Overview

Predict measured Rhomax wavelengths (nm) from amino-acid sequence on all 184 held-out archived test records. This selected split is not a complete FLIP2 score.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

Results

Each comparison retains its reviewed evaluation scope, dataset and metric. Results are shown without a pooled ranking.

FLIP2 Rhomax by_wild_type: local controls

ndcg (dimensionless) · Higher values are better.

Matched local controls on the same complete selected test split; not a comparison with published model scores.

FLIP2 Rhomax by_wild_type · FLIP2 Rhomax by_wild_type test subset

Evidence origin: Rewire evaluation. Numerical source review does not establish independent reproduction.

Amino-acid composition + fixed ridge: local execution report (20 September 2026); Training-mean control: local execution report (20 September 2026) · Execution reports: metrics.ndcg

One execution per configuration; uncertainty not estimated.

All comparison limitations (2)
  • One execution per configuration; uncertainty not estimated.
  • This selected protocol is not a benchmark-wide aggregate.

Automated source review: 2026-09-20.

No unavailable values; missing scores remain labelled and are never plotted as zero.

Showing 2 of 2 matching rows. Source order is preserved.

Tested configuration
00.250.50.751
Reported score
  1. Amino-acid composition + fixed ridge (FLIP2 Rhomax by_wild_type)0.954819941821582
  2. Training-mean control (FLIP2 Rhomax by_wild_type)0.9206667522227658

Dots show point estimates. Whiskers show only explicitly defined uncertainty (standard deviation, standard error or a labelled interval); their definitions remain in Table. Unresolved uncertainty is not plotted. Differences do not establish statistical significance.

Methods and evaluation design

Procedure, tasks and evaluated configurations

Evaluation design

Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.

Benchmarks

These source-backed links do not make different protocols or scores interchangeable.

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Run this benchmark

Repeat the four local sequence controls

Predict measured Rhomax wavelengths (nm) from amino-acid sequence on all 184 held-out archived test records. This selected split is not a complete FLIP2 score.

Generate predictions and evaluate them. This recipe does not establish reproduction of a particular published score.

Dataset access
Obtain the permitted source files and verify hashes following the cited instructions before running.
Model and weights
No pretrained weights; procedural controls fit only training labels.
Licences
Follow original dataset and software reuse terms. No dataset or weight redistribution is included.
Software
Pinned runner checkout; Python 3.11.13 and rewirebench 0.4.0 wheel. Install the exact executed dependency list linked in the instructions.
Hardware
Executed on macOS arm64 CPU, one thread. No accelerator required; memory and cross-platform performance unreported.
Required inputs and expected outputs

Inputs

  • Pinned Rhomax by_wild_type CSV and Sample designed parquet; script evaluates both datasets and both controls

Outputs

  • Private predictions, coverage report, metrics and sanitized contribution bundle

Execution steps

  1. 1. Run from the pinned runner checkout after preparation (Command line)

    Source reviewed; these instructions have not been executed by rewire.

    OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 python research/local-runs-2026-09-20/run_sequence.py \
  --flip-source local-inputs/by_wild_type.csv.gz \
  --mrna-source local-inputs/mrl-sample-designed.parquet \
  --work local-execution --evidence local-evidence
    FLIP2 Rhomax by_wild_type: reproduction instructions · Reproduce section; replace paths with prepared local inputs

Use your own model

Run your model locally and return predictions keyed by the input IDs. The evaluator supplies biological inputs without test labels and owns scoring. This interface is not a sandbox for model code.

Pass your existing prediction function into this adapter. Its output direction must match the selected protocol.

class MyModelAdapter:
    def __init__(self, score):
        self.score = score

    def predict(self, inputs):
        return {row["id"]: float(self.score(row)) for row in inputs}

# adapter = MyModelAdapter(your_prediction_function)
# report = rewirebench.run(prepared, adapter, output="runs/my-model")

Alternatively, generate a keyed prediction file in your existing model environment and use the score-only recipe. Your model code and weights do not need to be shared.

FLIP2 Rhomax by_wild_type: reproduction instructions · Reproduction instructions and execution scope
Scope and limitations
  • The script was executed locally, but these portable path examples have not been rerun verbatim.
  • Only the selected assay or selected sequence controls are evaluated; no suite aggregate.
  • Use new output directories. Raw predictions and private inputs are not uploaded.
  • Matching a fresh local result is distinct from reproducing an earlier paper score.

Contribute a result for review. The library can submit an exported evaluation for private review when intake is open. Check the contribution page for access and sign-in.

Original repository instructions

Run instructions

No runnable recipe has been reviewed for this protocol. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

Strengths, limitations and unresolved questions

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

7 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-22-f58a0f1d267f
Property and statementOriginal source and locationReview and provenance
Run recipe: inputs

["Pinned Rhomax by_wild_type CSV and Sample designed parquet; script evaluates both datasets and both controls"]

Individual claims		FLIP2 Rhomax by_wild_type: reproduction instructions

Original source ↗

Reproduction instructions and execution scope

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

source checked

implementation review

Audit details

Recipe metadata reviewed against pinned sources; not experimental reproduction.

Field: attributes.run_recipes.0.inputs

Source artifact SHA-256: 75f50703210dc556d77fdfe8f1d28c3200121b5b78c5c6592a096d928b4ff7cc

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Run recipe: Run from the pinned runner checkout after preparation

OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 python research/local-runs-2026-09-20/run_sequence.py \ --flip-source local-inputs/by_wild_type.csv.gz \ --mrna-source local-inputs/mrl-sample-designed.parquet \ --work local-execution --evidence local-evidence

Individual claims		FLIP2 Rhomax by_wild_type: reproduction instructions

Original source ↗

Reproduce section; replace paths with prepared local inputs

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

source checked

implementation review

Audit details

Execution status: source_reviewed_not_executed. Execution receipts apply only to their stated scope; not experimental reproduction.

Field: attributes.run_recipes.0.instructions.0.code

Source artifact SHA-256: 75f50703210dc556d77fdfe8f1d28c3200121b5b78c5c6592a096d928b4ff7cc

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Run recipe: limitations

["The script was executed locally, but these portable path examples have not been rerun verbatim.","Only the selected assay or selected sequence controls are evaluated; no suite aggregate.","Use new output directories. Raw predictions and private inputs are not uploaded.","Matching a fresh local result is distinct from reproducing an earlier paper score."]

Individual claims		FLIP2 Rhomax by_wild_type: reproduction instructions

Original source ↗

Reproduction instructions and execution scope

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

source checked

implementation review

Audit details

Recipe metadata reviewed against pinned sources; not experimental reproduction.

Field: attributes.run_recipes.0.limitations

Source artifact SHA-256: 75f50703210dc556d77fdfe8f1d28c3200121b5b78c5c6592a096d928b4ff7cc

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Run recipe: outputs

["Private predictions, coverage report, metrics and sanitized contribution bundle"]

Individual claims		FLIP2 Rhomax by_wild_type: reproduction instructions

Original source ↗

Reproduction instructions and execution scope

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

source checked

implementation review

Audit details

Recipe metadata reviewed against pinned sources; not experimental reproduction.

Field: attributes.run_recipes.0.outputs

Source artifact SHA-256: 75f50703210dc556d77fdfe8f1d28c3200121b5b78c5c6592a096d928b4ff7cc

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Run recipe: requirements

{"data":"Obtain the permitted source files and verify hashes following the cited instructions before running.","weights":"No pretrained weights; procedural controls fit only training labels.","licence":"Follow original dataset and software reuse terms. No dataset or weight redistribution is included.","software":"Pinned runner checkout; Python 3.11.13 and rewirebench 0.4.0 wheel. Install the exact executed dependency list linked in the instructions.","hardware":"Executed on macOS arm64 CPU, one thread. No accelerator required; memory and cross-platform performance unreported."}

Individual claims		FLIP2 Rhomax by_wild_type: reproduction instructions

Original source ↗

Reproduction instructions and execution scope

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

source checked

implementation review

Audit details

Recipe metadata reviewed against pinned sources; not experimental reproduction.

Field: attributes.run_recipes.0.requirements

Source artifact SHA-256: 75f50703210dc556d77fdfe8f1d28c3200121b5b78c5c6592a096d928b4ff7cc

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Run recipe: summary

Predict measured Rhomax wavelengths (nm) from amino-acid sequence on all 184 held-out archived test records. This selected split is not a complete FLIP2 score.

Individual claims		FLIP2 Rhomax by_wild_type: reproduction instructions

Original source ↗

Reproduction instructions and execution scope

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

source checked

implementation review

Audit details

Recipe metadata reviewed against pinned sources; not experimental reproduction.

Field: attributes.run_recipes.0.summary

Source artifact SHA-256: 75f50703210dc556d77fdfe8f1d28c3200121b5b78c5c6592a096d928b4ff7cc

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Relationship: part of

discovery-benchmark-flip2

Individual claims		Amino-acid composition + fixed ridge: local execution report (20 September 2026)

Original source ↗

protocol_id; protocol_configuration; provenance

Version: ca73fa47136d182f2d4ddb083d084712198fc0e2
Retrieved: 2026-09-20T20:54:03Z

source checked

automated execution evidence review · 2026-09-20T20:54:03Z

Audit details

Field: links:part_of:discovery-benchmark-flip2

Claim: rewire-protocol-flip2-rhomax-by-wild-type-v1-part-of-discovery-benchmark-flip2

Source artifact SHA-256: abd08d3f493428d06f43740c94e268575d755560e72120c73b294127acb395e6

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-22-f58a0f1d267f · Record review: source checked

2 source records and release historyDownload this release
Technical metadata and extraction receipts

Stable ID: rewire-protocol-flip2-rhomax-by-wild-type-v1

areas
proteins-complexes
protocol id
flip2-fitness-v1
version
1
procedure
adapter input contract: opaque-sequence-inputs-v1; canonical split counts: test: 184; train: 584; validation: 116; canonical test count: 184; data verification: pinned_source_bytes; dataset: rhomax; embedding probe: alpha: 10; feature scaling: none; fit intercept: true; max iter: 1000000; name: rewire-frozen-embedding-ridge-v1; refit: train only; relationship to published baseline: Frozen-embedding extension of upstream alpha-10 ridge. The published baseline uses one-hot features and scales targets using train plus validation rows. This extension is not that published baseline.; solver: auto; target scaling: StandardScaler fitted only on train labels; tol: 0.00001; validation use: none; fixed hyperparameters; input representation: amino-acid sequence; metric: spearman; metric direction: higher; metrics: spearman; ndcg; prepared rows sha256: d86fe3926529df72649f4638ffa0d89ec62460d831ceb62ad47ff86249469a12; smoke limit per split: Not reported; source note: Not reported; source reuse terms: CC-BY-4.0 archive; source attribution under resources/flip2; split: by_wild_type; split counts: test: 184; train: 584; validation: 116; suite complete: false; target units: nm; task: regression
source locator
protocol_configuration; protocol_results; provenance
access note
See the pinned execution report and source retrieval evidence. Dataset reuse terms are source-specific.
reproducibility note
Fresh local evaluation; reference metric agreement is not reproduction of a paper score.
reproduction url
https://github.com/rewire-bio/rewire-benchmarks/blob/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/README.md
run recipes
id: local-20260920-flip2-repeat; protocol id: rewire-protocol-flip2-rhomax-by-wild-type-v1; version: ca73fa47136d182f2d4ddb083d084712198fc0e2; title: Repeat the four local sequence controls; purpose: generate_and_evaluate; summary: Predict measured Rhomax wavelengths (nm) from amino-acid sequence on all 184 held-out archived test records. This selected split is not a complete FLIP2 score.; inputs: Pinned Rhomax by_wild_type CSV and Sample designed parquet; script evaluates both datasets and both controls; outputs: Private predictions, coverage report, metrics and sanitized contribution bundle; requirements: data: Obtain the permitted source files and verify hashes following the cited instructions before running.; weights: No pretrained weights; procedural controls fit only training labels.; licence: Follow original dataset and software reuse terms. No dataset or weight redistribution is included.; software: Pinned runner checkout; Python 3.11.13 and rewirebench 0.4.0 wheel. Install the exact executed dependency list linked in the instructions.; hardware: Executed on macOS arm64 CPU, one thread. No accelerator required; memory and cross-platform performance unreported.; instructions: runtime: command_line; title: Run from the pinned runner checkout after preparation; code: OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 python research/local-runs-2026-09-20/run_sequence.py \ --flip-source local-inputs/by_wild_type.csv.gz \ --mrna-source local-inputs/mrl-sample-designed.parquet \ --work local-execution --evidence local-evidence; status: source_reviewed_not_executed; source ids: rewire-local-20260920-instructions-flip2; source locator: Reproduce section; replace paths with prepared local inputs; limitations: The script was executed locally, but these portable path examples have not been rerun verbatim.; Only the selected assay or selected sequence controls are evaluated; no suite aggregate.; Use new output directories. Raw predictions and private inputs are not uploaded.; Matching a fresh local result is distinct from reproducing an earlier paper score.; source ids: rewire-local-20260920-instructions-flip2; source locator: Reproduction instructions and execution scope
comparison panels
id: local-20260920-flip2-ndcg; title: FLIP2 Rhomax by_wild_type: local controls; protocol id: rewire-protocol-flip2-rhomax-by-wild-type-v1; dataset id: rewire-dataset-flip2-rhomax-by-wild-type-v3; metric: ndcg; unit: dimensionless; direction: higher; result ids: rewire-local-20260920-result-flip2-composition-ndcg; rewire-local-20260920-result-flip2-train-mean-ndcg; source ids: rewire-local-20260920-source-flip2-composition; rewire-local-20260920-source-flip2-train-mean; source locator: Execution reports: metrics.ndcg; context: Matched local controls on the same complete selected test split; not a comparison with published model scores.; caveats: One execution per configuration; uncertainty not estimated.; This selected protocol is not a benchmark-wide aggregate.; review: method: automated_source_review; date: 2026-09-20
Related records

Suggest a correction