rewire.itbenchmarks

← Back to use cases

Cells and tissues · Use case

Assess models for genetic perturbation experiments

Which prediction methods and controls should I test before using expression predictions to plan genetic perturbation experiments?

Your decision and inputs

Choose models and controls for a pilot in the intended cell system before using predictions to select follow-up experiments. The current evidence compares expression prediction; it does not establish experimental hit rates.

Who this is for
Researchers designing genetic perturbation screens; Computational biologists evaluating perturbation-response models
Context
Research
Inputs
  • Single-cell expression measurements from perturbed and unperturbed cells, with perturbation and cell-type labels
  • A study design with multiple measured perturbations and cells per perturbation; combinations require combination examples during GEARS training
Expected output
Predicted post-perturbation expression and changes relative to unperturbed controls, assessed against measured responses.
Biological setting
Research method assessment anchored to the Norman2019 K562 cell-line comparison in GEARS Supplementary Table 6. Both endpoints below describe the same four evaluated configurations.

Outside this use case

  • Cross-cell-type transfer and bulk-sequencing prediction are outside the cited GEARS usage scope.
  • Training GEARS only on single-gene perturbations does not support reliable combination prediction.
  • Expression prediction scores do not establish causal mechanism, fitness effects or successful experimental prioritisation.

What this establishes for clinical research

Research use only. These expression endpoints do not establish diagnostic, treatment-selection or patient-response validity.

Which evaluations inform this question?

Evidence is grouped by its protocol. Relevance refers to the stated endpoint and context; it is separate from clinical validation and from the review method.

GEARS Norman2019 CPA-control comparison MSE: Norman2019 perturbation-response MSE

Current source-reviewed mapping

Proxy evidence: transfer to this question is limited

The complete comparison includes a no-change control, CPA, CPA with knowledge-graph features and GEARS. It can inform which controls to include in a pilot; aggregate expression agreement does not directly measure follow-up experiment yield.

Assessed endpoint
Mean squared error of predicted versus observed post-perturbation expression; lower is better.
Evaluation protocol
GEARS Norman2019 CPA-control comparison MSE: Norman2019 perturbation-response MSE
Computational task
A reviewed task relationship is not recorded for this protocol.
Input and population constraints
  • Use the exact Norman2019 K562 source-table protocol and configurations; do not combine this evidence with PerturBench or other GEARS splits.
  • Read both Table 6 endpoints together while retaining their separate metrics and shared experimental provenance.

Limits on interpretation

  • Exact Table 6 split and scored gene subset remain unextracted; checkpoint and runtime requirements are not established by these catalogue evaluations.
  • Reported spreads have unspecified type. A numerical difference is not a significance test.
  • Predictions need validation in the intended cell system before experiment selection.

Automated source review · 2026-09-28 · Codex research curation

Primary-source curation and separate automated cross-review checked exact evidence identities, comparator coverage, endpoint relevance and transfer limits. No new execution, human scientific review, independent replication or clinical validation.

Evaluated configurations

Each configuration below belongs to this protocol. Inspect its inputs, population and scoring conditions before comparing it with another evaluation.

No Perturb

Author-reported evaluation · Source checked

GEARS Norman2019 CPA-control comparison evaluation of No Perturb on Norman2019 perturbation-response MSE, scored with Mean squared error.

Population and split
Not reported · Not reported
Inputs and adaptation
Not reported · Not reported
Evaluation budget
Not reported
Runtime and memory

Runtime and memory measurements are not reported in this evaluation. A study budget is not a runtime or memory measurement.

Recorded results for this configuration
MetricValueCoverageUncertainty and source
mse0.551 ± 0.029

squared_expression_units_unreported · lower

Not reported scored / Not reported eligible

type: unreported; reported spread: 0.029; note: Printed ± spread; SD/SE/CI type not established by inspected table caption.

Result provenance

Evaluation methods, evidence and reproduction

No execution recipe has been verified for this exact configuration and evaluation. Inspect its methods and original run documentation before attempting reproduction.

CPA

Author-reported evaluation · Source checked

GEARS Norman2019 CPA-control comparison evaluation of CPA on Norman2019 perturbation-response MSE, scored with Mean squared error.

Population and split
Not reported · Not reported
Inputs and adaptation
Not reported · Not reported
Evaluation budget
Not reported
Runtime and memory

Runtime and memory measurements are not reported in this evaluation. A study budget is not a runtime or memory measurement.

Recorded results for this configuration
MetricValueCoverageUncertainty and source
mse0.354 ± 0.049

squared_expression_units_unreported · lower

Not reported scored / Not reported eligible

type: unreported; reported spread: 0.049; note: Printed ± spread; SD/SE/CI type not established by inspected table caption.

Result provenance

Evaluation methods, evidence and reproduction

No execution recipe has been verified for this exact configuration and evaluation. Inspect its methods and original run documentation before attempting reproduction.

CPA + KG

Author-reported evaluation · Source checked

GEARS Norman2019 CPA-control comparison evaluation of CPA + KG on Norman2019 perturbation-response MSE, scored with Mean squared error.

Population and split
Not reported · Not reported
Inputs and adaptation
Not reported · Not reported
Evaluation budget
Not reported
Runtime and memory

Runtime and memory measurements are not reported in this evaluation. A study budget is not a runtime or memory measurement.

Recorded results for this configuration
MetricValueCoverageUncertainty and source
mse0.333 ± 0.046

squared_expression_units_unreported · lower

Not reported scored / Not reported eligible

type: unreported; reported spread: 0.046; note: Printed ± spread; SD/SE/CI type not established by inspected table caption.

Result provenance

Evaluation methods, evidence and reproduction

No execution recipe has been verified for this exact configuration and evaluation. Inspect its methods and original run documentation before attempting reproduction.

GEARS

Author-reported evaluation · Source checked

GEARS Norman2019 CPA-control comparison evaluation of GEARS on Norman2019 perturbation-response MSE, scored with Mean squared error.

Population and split
Not reported · Not reported
Inputs and adaptation
Not reported · Not reported
Evaluation budget
Not reported
Runtime and memory

Runtime and memory measurements are not reported in this evaluation. A study budget is not a runtime or memory measurement.

Recorded results for this configuration
MetricValueCoverageUncertainty and source
mse0.216 ± 0.053

squared_expression_units_unreported · lower

Not reported scored / Not reported eligible

type: unreported; reported spread: 0.053; note: Printed ± spread; SD/SE/CI type not established by inspected table caption.

Result provenance

Evaluation methods, evidence and reproduction

No execution recipe has been verified for this exact configuration and evaluation. Inspect its methods and original run documentation before attempting reproduction.

Open the protocol's results and comparison checks →

Mapping sources and review metadata

Mapping use-case-map-gears-norman-table6-mse · revision 1

Initial applicability review: primary sources and separate automated cross-review support this exact protocol, its complete selected comparator group and the stated limits.

Reviewed evidence fingerprint c28653a778ad2953ce30515281cfbcbec052ef644991079b2d503d7ae92c22c2

GEARS Norman2019 CPA-control comparison Pearson DE: Norman2019 perturbation-response Pearson DE

Current source-reviewed mapping

Proxy evidence: transfer to this question is limited

The complete comparison includes a no-change control, CPA, CPA with knowledge-graph features and GEARS. It can inform which controls to include in a pilot; aggregate expression agreement does not directly measure follow-up experiment yield.

Assessed endpoint
Pearson correlation of predicted versus observed expression change from unperturbed controls; higher is better.
Evaluation protocol
GEARS Norman2019 CPA-control comparison Pearson DE: Norman2019 perturbation-response Pearson DE
Computational task
A reviewed task relationship is not recorded for this protocol.
Input and population constraints
  • Use the exact Norman2019 K562 source-table protocol and configurations; do not combine this evidence with PerturBench or other GEARS splits.
  • Read both Table 6 endpoints together while retaining their separate metrics and shared experimental provenance.

Limits on interpretation

  • Exact Table 6 split and scored gene subset remain unextracted; checkpoint and runtime requirements are not established by these catalogue evaluations.
  • Reported spreads have unspecified type. A numerical difference is not a significance test.
  • Predictions need validation in the intended cell system before experiment selection.

Automated source review · 2026-09-28 · Codex research curation

Primary-source curation and separate automated cross-review checked exact evidence identities, comparator coverage, endpoint relevance and transfer limits. No new execution, human scientific review, independent replication or clinical validation.

Evaluated configurations

Each configuration below belongs to this protocol. Inspect its inputs, population and scoring conditions before comparing it with another evaluation.

No Perturb

Author-reported evaluation · Source checked

GEARS Norman2019 CPA-control comparison evaluation of No Perturb on Norman2019 perturbation-response Pearson DE, scored with Pearson correlation of differential expression.

Population and split
Not reported · Not reported
Inputs and adaptation
Not reported · Not reported
Evaluation budget
Not reported
Runtime and memory

Runtime and memory measurements are not reported in this evaluation. A study budget is not a runtime or memory measurement.

Recorded results for this configuration
MetricValueCoverageUncertainty and source
pearson_delta_expression0.004 ± 0.006

correlation · higher

Not reported scored / Not reported eligible

type: unreported; reported spread: 0.006; note: Printed ± spread; SD/SE/CI type not established by inspected table caption.

Result provenance

Evaluation methods, evidence and reproduction

No execution recipe has been verified for this exact configuration and evaluation. Inspect its methods and original run documentation before attempting reproduction.

CPA

Author-reported evaluation · Source checked

GEARS Norman2019 CPA-control comparison evaluation of CPA on Norman2019 perturbation-response Pearson DE, scored with Pearson correlation of differential expression.

Population and split
Not reported · Not reported
Inputs and adaptation
Not reported · Not reported
Evaluation budget
Not reported
Runtime and memory

Runtime and memory measurements are not reported in this evaluation. A study budget is not a runtime or memory measurement.

Recorded results for this configuration
MetricValueCoverageUncertainty and source
pearson_delta_expression0.440 ± 0.036

correlation · higher

Not reported scored / Not reported eligible

type: unreported; reported spread: 0.036; note: Printed ± spread; SD/SE/CI type not established by inspected table caption.

Result provenance

Evaluation methods, evidence and reproduction

No execution recipe has been verified for this exact configuration and evaluation. Inspect its methods and original run documentation before attempting reproduction.

CPA + KG

Author-reported evaluation · Source checked

GEARS Norman2019 CPA-control comparison evaluation of CPA + KG on Norman2019 perturbation-response Pearson DE, scored with Pearson correlation of differential expression.

Population and split
Not reported · Not reported
Inputs and adaptation
Not reported · Not reported
Evaluation budget
Not reported
Runtime and memory

Runtime and memory measurements are not reported in this evaluation. A study budget is not a runtime or memory measurement.

Recorded results for this configuration
MetricValueCoverageUncertainty and source
pearson_delta_expression0.504 ± 0.029

correlation · higher

Not reported scored / Not reported eligible

type: unreported; reported spread: 0.029; note: Printed ± spread; SD/SE/CI type not established by inspected table caption.

Result provenance

Evaluation methods, evidence and reproduction

No execution recipe has been verified for this exact configuration and evaluation. Inspect its methods and original run documentation before attempting reproduction.

GEARS

Author-reported evaluation · Source checked

GEARS Norman2019 CPA-control comparison evaluation of GEARS on Norman2019 perturbation-response Pearson DE, scored with Pearson correlation of differential expression.

Population and split
Not reported · Not reported
Inputs and adaptation
Not reported · Not reported
Evaluation budget
Not reported
Runtime and memory

Runtime and memory measurements are not reported in this evaluation. A study budget is not a runtime or memory measurement.

Recorded results for this configuration
MetricValueCoverageUncertainty and source
pearson_delta_expression0.556 ± 0.030

correlation · higher

Not reported scored / Not reported eligible

type: unreported; reported spread: 0.030; note: Printed ± spread; SD/SE/CI type not established by inspected table caption.

Result provenance

Evaluation methods, evidence and reproduction

No execution recipe has been verified for this exact configuration and evaluation. Inspect its methods and original run documentation before attempting reproduction.

Open the protocol's results and comparison checks →

Mapping sources and review metadata

Mapping use-case-map-gears-norman-table6-pearson-de · revision 1

Initial applicability review: primary sources and separate automated cross-review support this exact protocol, its complete selected comparator group and the stated limits.

Reviewed evidence fingerprint 87737c4787ced71240c112f2f4cc859131132633b9f029abb0d8aa043f310486

What evidence is still missing?

  • The exact Table 6 split manifest, scoring gene subset, scored counts and checkpoint revisions remain unextracted. Do not silently label the MSE endpoint as Top20.
  • The table does not identify its printed spread as SD, SE or a confidence interval; these scores do not establish a statistically supported winner.
  • Both endpoints and the repeated No Perturb row are overlapping evidence from one source comparison, not independent replications.
  • Performance in a new cell system and experimental hit rates need a separate prospective evaluation. Human scientific review remains outstanding.

Contribute evidence or propose a correction

Sources and review

Automated source review · 2026-09-28 · Codex research curation

Primary-source curation and separate automated cross-review checked exact evidence identities, comparator coverage, endpoint relevance and transfer limits. No new execution, human scientific review, independent replication or clinical validation.

Release provenance and downloads

Release 2026-09-28-c7b5ac6d34f2

Use-case input digest a0dd27a5f430ec387d309fd6e8615083250873ce4cff5e882c6fb8458ef80e95

Download questions, collection plans and review metadata (JSON) · Verify release checksums

Question use-case-genetic-perturbation-response. Any numerical results on this page come from this release's existing evaluation records.