rewire.it
Task

Donor-held-out reference mapping

Donor-held-out reference mapping asks whether a reference learned from some donors remains useful for cells from a new donor.

Sources (2)scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

0 evaluations · 0 metric rows

At a glance

Inputs, training, access and other details

Explanatory profile: source reviewed · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.

Data, procedure and scoring
PropertyDescription and evidence
Entity typeTask guide; concrete protocol identities remain separate.
Sources (2)scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
DatasetsNo single dataset is fixed by this guide. Select a linked protocol and its versioned data release. · Not applicable
Sources (2)scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
OrganismsNo shared organism population is defined at this guide level. Record it for each selected dataset. · Not applicable
Sources (2)scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
AssaysNo single measurement assay is fixed by this guide; the endpoint and assay belong to the selected protocol. · Not applicable
Sources (2)scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
SplitsThis proposed guide requires donor separation; no particular donor assignment or cohort is fixed here. · Not applicable
Sources (2)scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
Allowed inputsReference and query molecular measurements, compatible feature identities, donor metadata and labels reserved for assessment.
Sources (2)scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
AdaptationNo common fitting regime is imposed here. Keep pretrained, frozen, probed, fine-tuned and conventional methods distinct where applicable. · Not applicable
Sources (2)scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
MetricsChoose label-transfer metrics for annotations and biological-conservation metrics for representations. The exact metric, averaging and unknown-cell handling belong to the concrete protocol.
Sources (2)scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
BaselinesSpecify a reference-only comparator with the same available labels and features. No measured baseline is assigned here.
Sources (2)scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

How it works

How it worksConceptual evaluation workflow
Conceptual evaluation workflow1. Define reference and query donors. Then: 2. Fit reference using permitted data. Then: 3. Map held-out query cells. Then: 4. Score labels and biological structureConceptual evaluation workflow1. Define reference and query donors. Then: 2. Fit reference using permitted data. Then: 3. Map held-out query cells. Then: 4. Score labels and biological structureConceptual evaluation workflow1. Define reference and query donors. Then: 2. Fit reference using permitted data. Then: 3. Map held-out query cells. Then: 4. Score labels and biological structure

Conceptual task guide. Dataset preparation, parameters and scoring must come from a separately identified protocol.

Sources (2)scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
Choosing an evaluation

Keep donor identity visible throughout the evaluation. The reference and any fitted annotation rules belong to the training data; a query donor supplies the held-out assessment. Cell representations and transferred labels are different outputs and require different scoring. This is a proposed task definition, not a claim that every reference-mapping paper uses donor holdout.

Sources (2)scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
Task scope

This is a task guide, not a single versioned benchmark protocol. The connected resources provide examples or concrete procedures. A candidate method or proposed control is not evidence that an evaluation has been completed.

Sources (2)scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

Tested entities and results

Release 2026-09-17-d277315f7d76 · 0 evaluations · 0 metric rows. Different protocols are not a single leaderboard.

No evaluations linked in this release.

Papers and result coverage

Last literature check: 2026-09-17. Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.

Paper or primary resourceVersionReference
Probabilistic harmonization and annotation of single‐cell transcriptomics data with deep generative modelsPMC full-text XML snapshot PMC7829634 at retrieval; byte-pinned by SHA-256Read source
DOI: 10.15252/msb.20209620
Defining and benchmarking open problems in single-cell analysisPMC full-text XML snapshot PMC11030530 at retrieval; byte-pinned by SHA-256Read source
DOI: 10.21203/rs.3.rs-4181617/v1
openproblems-label primary benchmark evidencev1.0.0Read source

What is still missing

  • A split by batch/reference is not automatically a donor-held-out split. A chart needs the exact dataset donor manifest.
  • Open Problems controls and normalization choices must remain separate from raw label accuracy/F1.
Search and extraction details

broad task primary protocol linked

Searches

  • Open Problems single cell label projection benchmark v1.0.0 paper
  • CAPRI assessment rounds 46 54 protein docking results
  • GEARS predicting transcriptional outcomes multigene perturbations 2023
  • scVI scANVI reference mapping benchmark primary paper
  • Boltz-2 affinity benchmark paper
  • ESMFold monomer structure benchmark Science 2023
  • "ProkBERT" "paper" "2024"
  • "scPertEval"
  • ProkBERT family prokaryotic language models microbe paper
  • Probabilistic harmonization annotation single-cell transcriptomics scANVI Nature Methods 2021
  • Evolutionary-scale prediction atomic-level protein structure language model ESMFold Science Lin 2023
  • Towards Principled Evaluation Single-Cell Perturbation Prediction Models Schäfer 2026

Evidence locations

  • scANVI Results and evaluation metrics
  • Open Problems: Label projection
  • Versioned label-projection task page

Strengths and limitations

Strengths supported by sources

  • Separating donors makes the intended generalisation question explicit and avoids confusing held-out cells with a new biological replicate.
    Sources (2)scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

Limitations and conditions

  • A donor split alone does not control every difference in assay, tissue or cell composition. This guide does not define a particular cohort.
    Sources (2)scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
Profile review details

Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol.

Stable record: catalog-task-cell-reference-mapping

Applicable tests and references

Applicability is distinct from a completed evaluation.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

34 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-17-d277315f7d76
Property and statementOriginal source and locationReview and provenance
Diagram caption

Conceptual task guide. Dataset preparation, parameters and scoring must come from a separately identified protocol.

Individual claims
scverse/scvi-tools: docs/user_guide/models/scvi.md

Original source ↗

scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 73b28e44223621470e582a81a102c107bb22678b
Retrieved: 2026-09-16T19:46:20.137176+00:00

source checked

automated source review · 2026-09-16

Audit details

Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol.

Field: attributes.profile.diagram.caption

Source artifact SHA-256: c81a5cfb30db1eca8c03af3d0db1a36a447a1bb5a27b270de5633ba73ebeeeef

Hash scope: SHA-256 of retrieved original artifact bytes

Format: original_artifact

Inspected artifact

Diagram caption

Conceptual task guide. Dataset preparation, parameters and scoring must come from a separately identified protocol.

Individual claims
theislab/scib official source

Original source ↗

scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: cd67913396b4c0430710b3d90f1d1841f5fa4468
Retrieved: 2026-09-16T10:31:32.426513+00:00

source checked

automated source review · 2026-09-16

Audit details

Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol.

Field: attributes.profile.diagram.caption

Source artifact SHA-256: db7aa3a701778d541bf47d8214b50e1ce9f73e92dc3e810bfd740270e9e353aa

Hash scope: Hash scope not separately documented; inspect source record

Diagram steps

["Define reference and query donors","Fit reference using permitted data","Map held-out query cells","Score labels and biological structure"]

Individual claims
scverse/scvi-tools: docs/user_guide/models/scvi.md

Original source ↗

scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 73b28e44223621470e582a81a102c107bb22678b
Retrieved: 2026-09-16T19:46:20.137176+00:00

source checked

automated source review · 2026-09-16

Audit details

Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol.

Field: attributes.profile.diagram.steps

Source artifact SHA-256: c81a5cfb30db1eca8c03af3d0db1a36a447a1bb5a27b270de5633ba73ebeeeef

Hash scope: SHA-256 of retrieved original artifact bytes

Format: original_artifact

Inspected artifact

Diagram steps

["Define reference and query donors","Fit reference using permitted data","Map held-out query cells","Score labels and biological structure"]

Individual claims
theislab/scib official source

Original source ↗

scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: cd67913396b4c0430710b3d90f1d1841f5fa4468
Retrieved: 2026-09-16T10:31:32.426513+00:00

source checked

automated source review · 2026-09-16

Audit details

Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol.

Field: attributes.profile.diagram.steps

Source artifact SHA-256: db7aa3a701778d541bf47d8214b50e1ce9f73e92dc3e810bfd740270e9e353aa

Hash scope: Hash scope not separately documented; inspect source record

Diagram title

Conceptual evaluation workflow

Individual claims
scverse/scvi-tools: docs/user_guide/models/scvi.md

Original source ↗

scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 73b28e44223621470e582a81a102c107bb22678b
Retrieved: 2026-09-16T19:46:20.137176+00:00

source checked

automated source review · 2026-09-16

Audit details

Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol.

Field: attributes.profile.diagram.title

Source artifact SHA-256: c81a5cfb30db1eca8c03af3d0db1a36a447a1bb5a27b270de5633ba73ebeeeef

Hash scope: SHA-256 of retrieved original artifact bytes

Format: original_artifact

Inspected artifact

Diagram title

Conceptual evaluation workflow

Individual claims
theislab/scib official source

Original source ↗

scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: cd67913396b4c0430710b3d90f1d1841f5fa4468
Retrieved: 2026-09-16T10:31:32.426513+00:00

source checked

automated source review · 2026-09-16

Audit details

Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol.

Field: attributes.profile.diagram.title

Source artifact SHA-256: db7aa3a701778d541bf47d8214b50e1ce9f73e92dc3e810bfd740270e9e353aa

Hash scope: Hash scope not separately documented; inspect source record

Entity type

Task guide; concrete protocol identities remain separate.

Individual claims
scverse/scvi-tools: docs/user_guide/models/scvi.md

Original source ↗

scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 73b28e44223621470e582a81a102c107bb22678b
Retrieved: 2026-09-16T19:46:20.137176+00:00

source checked

automated source review · 2026-09-16

Audit details

Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol.

Field: attributes.profile.facts.0.value

Source artifact SHA-256: c81a5cfb30db1eca8c03af3d0db1a36a447a1bb5a27b270de5633ba73ebeeeef

Hash scope: SHA-256 of retrieved original artifact bytes

Format: original_artifact

Inspected artifact

Entity type

Task guide; concrete protocol identities remain separate.

Individual claims
theislab/scib official source

Original source ↗

scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: cd67913396b4c0430710b3d90f1d1841f5fa4468
Retrieved: 2026-09-16T10:31:32.426513+00:00

source checked

automated source review · 2026-09-16

Audit details

Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol.

Field: attributes.profile.facts.0.value

Source artifact SHA-256: db7aa3a701778d541bf47d8214b50e1ce9f73e92dc3e810bfd740270e9e353aa

Hash scope: Hash scope not separately documented; inspect source record

Datasets

No single dataset is fixed by this guide. Select a linked protocol and its versioned data release.

Individual claims
scverse/scvi-tools: docs/user_guide/models/scvi.md

Original source ↗

scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 73b28e44223621470e582a81a102c107bb22678b
Retrieved: 2026-09-16T19:46:20.137176+00:00

inapplicable

automated source review · 2026-09-16

Audit details

Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol.

Field: attributes.profile.facts.1.value

Source artifact SHA-256: c81a5cfb30db1eca8c03af3d0db1a36a447a1bb5a27b270de5633ba73ebeeeef

Hash scope: SHA-256 of retrieved original artifact bytes

Format: original_artifact

Inspected artifact

Datasets

No single dataset is fixed by this guide. Select a linked protocol and its versioned data release.

Individual claims
theislab/scib official source

Original source ↗

scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: cd67913396b4c0430710b3d90f1d1841f5fa4468
Retrieved: 2026-09-16T10:31:32.426513+00:00

inapplicable

automated source review · 2026-09-16

Audit details

Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol.

Field: attributes.profile.facts.1.value

Source artifact SHA-256: db7aa3a701778d541bf47d8214b50e1ce9f73e92dc3e810bfd740270e9e353aa

Hash scope: Hash scope not separately documented; inspect source record

Sources and history

Release 2026-09-17-d277315f7d76 · Record review: discovered

9 source records and release historyDownload this release
Technical metadata and extraction receipts

Stable ID: catalog-task-cell-reference-mapping

areas
cells-tissues
entity level
task
version
Not reported
task
Donor-held-out reference mapping
scope note
Map unseen donors to a labelled cell-type reference.
benchmark research
review date: 2026-09-17; status: broad_task_primary_protocol_linked; primary sources: expansion-p3-scanvi; expansion-p3-open-problems-paper; expansion-p3-open-problems; inspected locators: scANVI Results and evaluation metrics; Open Problems: Label projection; Versioned label-projection task page; searched queries: Open Problems single cell label projection benchmark v1.0.0 paper; CAPRI assessment rounds 46 54 protein docking results; GEARS predicting transcriptional outcomes multigene perturbations 2023; scVI scANVI reference mapping benchmark primary paper; Boltz-2 affinity benchmark paper; ESMFold monomer structure benchmark Science 2023; "ProkBERT" "paper" "2024"; "scPertEval"; ProkBERT family prokaryotic language models microbe paper; Probabilistic harmonization annotation single-cell transcriptomics scANVI Nature Methods 2021; Evolutionary-scale prediction atomic-level protein structure language model ESMFold Science Lin 2023; Towards Principled Evaluation Single-Cell Perturbation Prediction Models Schäfer 2026; gaps: A split by batch/reference is not automatically a donor-held-out split. A chart needs the exact dataset donor manifest.; Open Problems controls and normalization choices must remain separate from raw label accuracy/F1.; claim scope: Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.
historical missing metadata
protocol version: not_yet_extracted
metadata review scope
historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
legacy kinds
benchmark
entity classification
review date: 2026-09-17; rationale: This record identifies the biological prediction question or a suite-specific task, rather than a uniquely fixed evaluated procedure. Preserve its task identity and leave split, model adaptation and scoring details on linked protocols/evaluations.; source ids: evidence-official-0cbab80c2d1769287a1f; src-discovery-theislab-scib; source locator: scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.; ambiguities: None recorded
Related records

Suggest a correction