ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire) on FLIP2 Rhomax by_wild_type
Frozen features with a fixed alpha-10 ridge head fitted only on training rows; all 184 Rhomax by_wild_type held-out proteins scored. This selected split is not a whole FLIP2 score.
Evaluation results
1 evaluation · 2 metric rows. Different protocols are not a single leaderboard.
Filter evaluations
Applied filters: All linked evaluations
Source checking is not independent reproduction. Release 2026-09-22-f58a0f1d267f.
Evaluation procedure
FLIP2 Rhomax by_wild_type
- Configuration
- ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire) (FLIP2 Rhomax by_wild_type)
- Protocol
- FLIP2 Rhomax by_wild_type
- Dataset subset
- FLIP2 Rhomax by_wild_type test subset
- origin
- Rewire evaluation
- configuration
- 1
- protocol id
- rewire-protocol-flip2-rhomax-by-wild-type-v1
- dataset version
- rewire-dataset-flip2-rhomax-by-wild-type-v3
- split
- canonical test
- subset
- flip2-rhomax-by-wild-type
- population
- 184/184
- inputs
- Complete amino-acid sequences; no labels enter encoder; no MSA/templates
- adaptation
- fixed train-only ridge on frozen representations
- metric implementation
- 9350e367b20ec48d9ce1195cb5b73c4bd98ce8530a2d1f7ba2157fc8be75fa34
- aggregation
- all held-out test rows of the selected target
- budget
- one local CPU evaluation; no held-out hyperparameter selection
Metadata review: source checked. Unreported conditions prevent automatic comparisons.
Reproduction
- Split
- canonical test
- Adaptation
- fixed train-only ridge on frozen representations
- Scoring implementation
- 9350e367b20ec48d9ce1195cb5b73c4bd98ce8530a2d1f7ba2157fc8be75fa34
No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.
Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
363 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| attributes.comparison.adaptation fixed train-only ridge on frozen representations Context-only references | ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): audit metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 | not individually reviewed No individual claim review recorded rewire run Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| attributes.comparison.adaptation fixed train-only ridge on frozen representations Context-only references | ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): execution and verification instructions metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 | not individually reviewed No individual claim review recorded rewire run Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| attributes.comparison.adaptation fixed train-only ridge on frozen representations Context-only references | ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): report metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 | not individually reviewed No individual claim review recorded rewire run Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| attributes.comparison.aggregation all held-out test rows of the selected target Context-only references | ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): audit metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 | not individually reviewed No individual claim review recorded rewire run Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| attributes.comparison.aggregation all held-out test rows of the selected target Context-only references | ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): execution and verification instructions metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 | not individually reviewed No individual claim review recorded rewire run Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| attributes.comparison.aggregation all held-out test rows of the selected target Context-only references | ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): report metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 | not individually reviewed No individual claim review recorded rewire run Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| attributes.comparison.budget one local CPU evaluation; no held-out hyperparameter selection Context-only references | ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): audit metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 | not individually reviewed No individual claim review recorded rewire run Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| attributes.comparison.budget one local CPU evaluation; no held-out hyperparameter selection Context-only references | ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): execution and verification instructions metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 | not individually reviewed No individual claim review recorded rewire run Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| attributes.comparison.budget one local CPU evaluation; no held-out hyperparameter selection Context-only references | ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): report metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 | not individually reviewed No individual claim review recorded rewire run Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| attributes.comparison.dataset_version rewire-dataset-flip2-rhomax-by-wild-type-v3 Context-only references | ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): audit metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 | not individually reviewed No individual claim review recorded rewire run Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
View linked audit checks and correction history
Release 2026-09-22-f58a0f1d267f · Record review: source checked
3 source records and release history
- ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): report · Original source · 1663d1f04b2bbd6dfcff77fea78129d30b0de191
- ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): audit · Original source · 1663d1f04b2bbd6dfcff77fea78129d30b0de191
- ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): execution and verification instructions · Original source · 1663d1f04b2bbd6dfcff77fea78129d30b0de191
Technical metadata and extraction receipts
Stable ID: rewire-local-20260921-evaluation-esm2-8m
- areas
- proteins-complexes
- origin
- rewire_run
- execution scope
- complete_selected_evaluation
- published score reproduction
- false
- protocol
- FLIP2 Rhomax by_wild_type
- version
- 1
- source locator
- metrics; coverage; protocol_results
- original sdk scope
- full
- original sdk completion
- complete
- suite complete
- false
- eligible count
- 184
- scored count
- 184
- missing count
- 0
- coverage
- 184/184
- comparison
- protocol id: rewire-protocol-flip2-rhomax-by-wild-type-v1; dataset version: rewire-dataset-flip2-rhomax-by-wild-type-v3; split: canonical test; subset: flip2-rhomax-by-wild-type; population: 184/184; inputs: Complete amino-acid sequences; no labels enter encoder; no MSA/templates; adaptation: fixed train-only ridge on frozen representations; metric implementation: 9350e367b20ec48d9ce1195cb5b73c4bd98ce8530a2d1f7ba2157fc8be75fa34; aggregation: all held-out test rows of the selected target; budget: one local CPU evaluation; no held-out hyperparameter selection
- input information
- Complete amino-acid sequences; no labels enter encoder; no MSA/templates
- model configuration
- checkpoint: esm2_t6_8M_UR50D; device: cpu; dimension: 320; encoder frozen: true; head: alpha: 10; feature scaling: none; fit intercept: true; max iter: 1000000; name: rewire-frozen-embedding-ridge-v1; refit: train only; relationship to published baseline: Frozen-embedding extension of upstream alpha-10 ridge. The published baseline uses one-hot features and scales targets using train plus validation rows. This extension is not that published baseline.; solver: auto; target scaling: StandardScaler fitted only on train labels; tol: 0.00001; validation use: none; fixed hyperparameters; pooling: mean over residues excluding BOS/EOS and padding; representation layer: 6; scope: one complete Rhomax split, not FLIP2 suite; torch seed: 0; torch threads: 4; validation used: false
- protocol configuration
- adapter input contract: opaque-sequence-inputs-v1; canonical split counts: test: 184; train: 584; validation: 116; canonical test count: 184; data verification: pinned_source_bytes; dataset: rhomax; embedding probe: alpha: 10; feature scaling: none; fit intercept: true; max iter: 1000000; name: rewire-frozen-embedding-ridge-v1; refit: train only; relationship to published baseline: Frozen-embedding extension of upstream alpha-10 ridge. The published baseline uses one-hot features and scales targets using train plus validation rows. This extension is not that published baseline.; solver: auto; target scaling: StandardScaler fitted only on train labels; tol: 0.00001; validation use: none; fixed hyperparameters; input representation: amino-acid sequence; metric: spearman; metric direction: higher; metrics: spearman; ndcg; prepared rows sha256: d86fe3926529df72649f4638ffa0d89ec62460d831ceb62ad47ff86249469a12; smoke limit per split: Not reported; source note: Not reported; source reuse terms: CC-BY-4.0 archive; source attribution under resources/flip2; split: by_wild_type; suite complete: false; target units: nm; task: regression
- provenance
- data verification: pinned_source_bytes; dataset version: zenodo-18433203-v3; evaluator url: https://github.com/J-SNACKKB/FLIP/blob/62cace8735f5610e2743cf06ce0f944b37fffaa6/baselines/aggregate.py#L131-L135; source csv sha256: 8e78f6a16cd5298131dca83130d4b94cc0ab4ae9c699460880441c19a65f656f; source sha256: 7c6d2f02cb89310378ac9897c321fbbd909fb0757ac88803dcce84f5f6b05ce3; source url: https://zenodo.org/api/records/18433203/files/rhomax/by_wild_type.csv.gz/content; split origin: Archived assignments: validation=True held out of training; upstream revision: 62cace8735f5610e2743cf06ce0f944b37fffaa6
- predictions sha256
- b975eff8d023c310860b6d8215fbe365a694985950afc95966db632eaa5eeb6c
- prepared sha256
- f417d43497e0c8c453bd7c66cfea05959b4d6d47462283096e42a2dae8b72419
- execution
- adapter: ESM2Embeddings; adapter provenance: checkpoint sha256: 46f002a9870c9bdecd0ea887acb1f9a38a6b561e8f8bf8a6990b679b9d31b928; checkpoint url: https://dl.fbaipublicfiles.com/fair-esm/models/esm2_t6_8M_UR50D.pt; device: cpu; fair esm version: 2.0.0; implementation sha256: 1f475a084964698145fb07f7d0cd851a486d06a508e266904160d6d6ec9ef9bb; maximum sequence length: 1022; model: esm2_t6_8M_UR50D; published result reproduction: false; reference: https://github.com/facebookresearch/esm#usage; representation layer: 6; strategy: frozen_final_layer_residue_mean_excluding_special_tokens; training overlap: unreported; UniRef50 pretraining may overlap benchmark proteins; batch size: 4; fitting: protocol_owned_probe; inference and fit seconds: 17.59812141599832; mode: local_adapter; prediction type: embedding
- timing seconds
- evaluation: 0.048134666998521425
- environment
- architecture: arm64; container digest: unreported; container runtime: unreported; platform: macOS-26.6.2-arm64-arm-64bit; python: 3.11.13; rewirebench: 0.4.0; sdk code sha256: 9350e367b20ec48d9ce1195cb5b73c4bd98ce8530a2d1f7ba2157fc8be75fa34; sif sha256: unreported
- run url
- https://github.com/rewire-bio/rewire-benchmarks/blob/1663d1f04b2bbd6dfcff77fea78129d30b0de191/research/baseline-programme-2026-09-21/rhomax/esm2.report.json
- reproduction url
- https://github.com/rewire-bio/rewire-benchmarks/blob/1663d1f04b2bbd6dfcff77fea78129d30b0de191/research/baseline-programme-2026-09-21/README.md
- reproduction note
- The linked pinned scripts describe this exact execution. No generic suite recipe is claimed to reproduce it; this review only checked existing artifacts.
- limitations
- No confidence intervals or seed variability estimate.; Source checking and local execution do not reproduce a published paper score.; Pretraining overlap is unreported; conclusions apply to the exact evaluated configuration.
Related records
- configuration: ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire) (FLIP2 Rhomax by_wild_type)
- protocol: FLIP2 Rhomax by_wild_type
- dataset: FLIP2 Rhomax by_wild_type test subset
- evaluation: ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): ndcg
- evaluation: ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): spearman