rewire.itbenchmarks

← Back to use cases

Cells and tissues · Use case

Transfer cell-type annotations to a new dataset

Which annotation workflow can label my new dataset reliably and recognise unsupported cell populations?

Evidence collection plan

Collection planned

Evidence collection is planned for this question. The plan defines a comparison to investigate; it does not establish model performance or suitability.

Comparison question

Does a method improve known-type annotation and unknown-type recognition at matched coverage over conventional methods across independent studies?

Baselines to include

  • Marker-based and nearest-reference annotation
  • Appropriate pinned Azimuth, CellTypist or scANVI workflows
  • Embeddings with a simple classifier where applicable

Outcomes to measure

  • Class-specific and hierarchy-aware errors
  • Rare-type and absent-reference performance
  • Calibration and accuracy as uncertain cells are left unassigned
  • Resource use and transfer across studies and platforms

Validation requirements

  • Hold out donors, whole studies and platforms as required by the intended query.
  • Audit duplicated cells, donors and labels across references and pretraining data.
  • Use independent expert or orthogonal label checks and preserve uncertainty in reference labels.
  • Match ontology level and coverage; keep integration and disease-state discovery separate.

Next collection task

Build an annotation-transfer evidence table with reference versions, donor/study independence, label hierarchy and coverage-aware outcomes.

Your decision and inputs

Choose an annotation workflow and reference, accept supported labels, and identify cells that require expert review or additional measurements.

Who this is for
Single-cell analysts; Cell atlas researchers
Context
Research
Inputs
  • Query count data with tissue, disease, donor and assay metadata
  • A versioned reference and a defined cell-label hierarchy
  • Independent marker, protein or expert evidence where available
  • Requirements for coverage, uncertainty and resource use
Expected output
Cell-type labels at the requested ontology level, confidence estimates and an explicit unassigned group.
Biological setting
Research annotation of a new single-cell or single-nucleus cohort. Transfer across donors, studies, disease contexts and technologies is assessed separately.

Outside this use case

  • Atlas labels are not infallible ground truth.
  • Embedding separation and batch mixing do not establish correct cell identity.
  • Annotation, batch integration and discovery of new disease states require separate evaluations.

What this establishes for clinical research

Research only. Annotation accuracy does not establish the validity of a clinical diagnostic classifier.

Which evaluations inform this question?

No model comparison has been collected for this question yet.

Relevant methods and studies may exist outside this collection.

View the evidence plan and next collection task

What evidence is still missing?

  • Comparisons need independent study and platform holdouts with reference and pretraining provenance.
  • Rare and absent-reference populations need explicit evaluation alongside common cell types.
  • Human domain review remains unassigned.

Contribute evidence or propose a correction

Sources and review

Automated source review · 2026-09-28 · Codex

Automated review of the workflow definition, cited primary-source scope and collection plan. No model evaluation or applicability mapping was added. Human domain review remains unassigned.

Release provenance and downloads

Release 2026-09-28-c7b5ac6d34f2

Use-case input digest a0dd27a5f430ec387d309fd6e8615083250873ce4cff5e882c6fb8458ef80e95

Download questions, collection plans and review metadata (JSON) · Verify release checksums

Question use-case-cell-type-annotation-transfer. Any numerical results on this page come from this release's existing evaluation records.