Cell-type annotation separates cross-batch prediction from a within-study random split.
Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Datasets | hPancreas, PBMC and Aorta annotated single-cell datasets.SourcesscELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis · Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 |
| Splits | hPancreas/PBMC withhold one batch; Aorta uses an 80:20 within-study split.SourcesscELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis · Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 |
| Metrics | Accuracy, precision, recall and F1.SourcesscELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis · Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 |
| Baselines | GPT-based classifiers, scGPT, Geneformer, GPTCelltype, MLP and PCA-derived representations.SourcesscELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis · Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 |
| Leakage controls | Batch-held-out evaluation is explicit for two datasets; it is not the split used for Aorta.SourcesscELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis · Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 |
| Uncertainty | Table 1 reports point metrics for annotation and identifies some rows as copied from GenePT. The cell-annotation section and table do not give repeated-run uncertainty or confidence intervals for the scELMo rows; copied comparator results must not be counted as independent replications. · Not reported in inspected sourcesSourcesscELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis · Results: scELMo for cell-type annotation; Table 1 caption |
| Entity type | Paper-specific computational evaluation protocol.SourcesscELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis · Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 |
| Organisms | Human pancreas, PBMC and Aorta datasets.SourcesscELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis · Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 |
| Assays | Single-cell expression and cell-type labels.SourcesscELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis · Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 |
| Allowed inputs | Expression-derived representations and GPT-based gene information.SourcesscELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis · Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 |
| Adaptation | Supervised annotation and classifier comparisons, with batch holdout where available.SourcesscELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis · Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 |
Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.
hPancreas, PBMC and Aorta annotated single-cell datasets. hPancreas/PBMC withhold one batch; Aorta uses an 80:20 within-study split. Accuracy, precision, recall and F1. GPT-based classifiers, scGPT, Geneformer, GPTCelltype, MLP and PCA-derived representations. Batch-held-out evaluation is explicit for two datasets; it is not the split used for Aorta.
Each evaluation records what was tested and under which conditions.
Release 2026-09-17-d277315f7d76 · 2 evaluations · 2 metric rows. Different protocols are not a single leaderboard.
| Metric and finding | Coverage and uncertainty | Evidence |
|---|---|---|
| scGPT: Cell-type annotation Zero-shot setting; source caption says some comparator rows come from GenePT. Result quoted from another source · Evaluation metadata: needs review | ||
| 0.550 F1 Unit: unitless · Direction: unknown | Uncertainty: not reported in legacy extract Scored: Not reported · Eligible: Not reported | source checkedscELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis · Table 1, hPancreas zero-shot / scGPT (z) row, F1 column Source checking is not independent reproduction. |
| Geneformer: Cell-type annotation Zero-shot setting; source caption says some comparator rows come from GenePT. Result quoted from another source · Evaluation metadata: needs review | ||
| 0.270 F1 Unit: unitless · Direction: unknown | Uncertainty: not reported in legacy extract Scored: Not reported · Eligible: Not reported | source checkedscELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis · Table 1, hPancreas zero-shot / Geneformer (z) row, F1 column Source checking is not independent reproduction. |
Last literature check: 2026-09-17. Dated primary-source discovery and protocol/table screening. Source checking does not mean experimental reproduction. Only separately extracted and independently reviewed numeric batches are publishable.
| Paper or primary resource | Version | Reference |
|---|---|---|
| scELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis | preprint archived 2025-08-23 | Read source |
primary comparison tables located
Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change.
Stable record: reported-task-660753ec94e631Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
18 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Diagram caption Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol. Individual claims | scELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 Version: preprint archived 2025-08-23 | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram steps ["Input: Expression-derived representations and GPT-based gene information.","Evaluation: hPancreas/PBMC withhold one batch; Aorta uses an 80:20 within-study split.","Readout: Accuracy, precision, recall and F1."] Individual claims | scELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 Version: preprint archived 2025-08-23 | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Computational evaluation flow Individual claims | scELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 Version: preprint archived 2025-08-23 | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Datasets hPancreas, PBMC and Aorta annotated single-cell datasets. Individual claims | scELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 Version: preprint archived 2025-08-23 | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Splits hPancreas/PBMC withhold one batch; Aorta uses an 80:20 within-study split. Individual claims | scELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 Version: preprint archived 2025-08-23 | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Adaptation Supervised annotation and classifier comparisons, with batch holdout where available. Individual claims | scELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 Version: preprint archived 2025-08-23 | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Metrics Accuracy, precision, recall and F1. Individual claims | scELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 Version: preprint archived 2025-08-23 | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Baselines GPT-based classifiers, scGPT, Geneformer, GPTCelltype, MLP and PCA-derived representations. Individual claims | scELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 Version: preprint archived 2025-08-23 | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Leakage controls Batch-held-out evaluation is explicit for two datasets; it is not the split used for Aorta. Individual claims | scELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis Results: Cell-type annotation; Methods: evaluation and baselines; cached text lines 23, 81, 96 Version: preprint archived 2025-08-23 | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Uncertainty Table 1 reports point metrics for annotation and identifies some rows as copied from GenePT. The cell-annotation section and table do not give repeated-run uncertainty or confidence intervals for the scELMo rows; copied comparator results must not be counted as independent replications. Individual claims | scELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis Results: scELMo for cell-type annotation; Table 1 caption Version: preprint archived 2025-08-23 | unreported automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Release 2026-09-17-d277315f7d76 · Record review: needs review
Stable ID: reported-task-660753ec94e631