Task

Community profiling

Community profiling estimates which microorganisms are present in a sample and their relative abundances.

Sources

CAMI-challenge/OPAL official source · README: Overview, Computed metrics and Inputs

At a glance

Inputs, training, access and other details

Explanatory profile: source reviewed · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.

Data, procedure and scoring
Property	Description and evidence
Entity type	Task guide; concrete protocol identities remain separate. Sources CAMI-challenge/OPAL official source · README: Overview, Computed metrics and Inputs
Datasets	No single dataset is fixed by this guide. Select a linked protocol and its versioned data release. · Not applicable Sources CAMI-challenge/OPAL official source · README: Overview, Computed metrics and Inputs
Organisms	No shared organism population is defined at this guide level. Record it for each selected dataset. · Not applicable Sources CAMI-challenge/OPAL official source · README: Overview, Computed metrics and Inputs
Assays	No single measurement assay is fixed by this guide; the endpoint and assay belong to the selected protocol. · Not applicable Sources CAMI-challenge/OPAL official source · README: Overview, Computed metrics and Inputs
Splits	No executable split is attached to this task identity. Use the selected protocol’s split manifest. · Not applicable Sources CAMI-challenge/OPAL official source · README: Overview, Computed metrics and Inputs
Allowed inputs	Predicted and reference taxonomic profiles with consistent taxon identifiers and abundance definitions. Sources CAMI-challenge/OPAL official source · README: Overview, Computed metrics and Inputs
Adaptation	No common fitting regime is imposed here. Keep pretrained, frozen, probed, fine-tuned and conventional methods distinct where applicable. · Not applicable Sources CAMI-challenge/OPAL official source · README: Overview, Computed metrics and Inputs
Metrics	OPAL reports precision, recall, F1, abundance errors such as L1 and Bray–Curtis, and diversity measures. These are distinct endpoints. Sources CAMI-challenge/OPAL official source · README: Overview, Computed metrics and Inputs
Baselines	Compare profilers against the same reference profile and taxonomic ranks; OPAL is an evaluator, not a predictive baseline. Sources CAMI-challenge/OPAL official source · README: Overview, Computed metrics and Inputs

How it works

How it worksConceptual evaluation workflow

Conceptual task guide. Dataset preparation, parameters and scoring must come from a separately identified protocol.

Sources

CAMI-challenge/OPAL official source · README: Overview, Computed metrics and Inputs

Choosing an evaluation

Compare a predicted taxonomic profile with a specified reference profile at declared taxonomic ranks. OPAL separates detection errors from abundance errors and diversity summaries. The reference taxonomy, filtering and abundance normalisation must accompany a result; changing these can change what is scored.

Sources

CAMI-challenge/OPAL official source · README: Overview, Computed metrics and Inputs

Task scope

This is a task guide, not a single versioned benchmark protocol. The connected resources provide examples or concrete procedures. A candidate method or proposed control is not evidence that an evaluation has been completed.

Sources

CAMI-challenge/OPAL official source · README: Overview, Computed metrics and Inputs

Tested entities and results

Release 2026-09-17-d277315f7d76 · 0 evaluations · 0 metric rows. Different protocols are not a single leaderboard.

No evaluations linked in this release.

Papers and result coverage

Last literature check: 2026-09-17. Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.

Paper or primary resource	Version	Reference
Critical Assessment of Metagenome Interpretation: the second round of challenges	PMC9007738	Read source DOI: 10.1038/s41592-022-01431-4
Assessing taxonomic metagenome profilers with OPAL	PMC6398228	Read source DOI: 10.1186/s13059-019-1646-y

What is still missing

OPAL is an evaluator rather than a single dataset. Numerical comparisons require selected community dataset, taxonomic rank and profiling output version. No complete main-text score matrix; source data remains pending.

Search and extraction details

broad task requires concrete protocol links

Searches

Community profiling primary paper benchmark results

Evidence locations

Table1 rankings; main figures and linked supplementary material
Methods and benchmark Results; figure captions and source availability

Strengths and limitations

Strengths supported by sources

Separate presence and abundance metrics reveal different failure modes instead of hiding them behind a single accuracy number.
Sources
CAMI-challenge/OPAL official source · README: Overview, Computed metrics and Inputs

Limitations and conditions

A profile-level result is not a read-binning result. Reference coverage and taxonomic rank affect interpretation.
Sources
CAMI-challenge/OPAL official source · README: Overview, Computed metrics and Inputs

Profile review details

Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol.

Stable record: catalog-task-community-profiling

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

17 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-17-d277315f7d76
Property and statement	Original source and location	Review and provenance
Diagram caption Conceptual task guide. Dataset preparation, parameters and scoring must come from a separately identified protocol. Individual claims	CAMI-challenge/OPAL official source Original source ↗ README: Overview, Computed metrics and Inputs Version: 98120c326eef08e391899e4bd3a362e0e6558b4a Retrieved: 2026-09-16T10:30:23.429372+00:00	source checked automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.diagram.caption` Source artifact SHA-256: `d123b202234d3ed520911982dcf88e2fcfd888c83e65378effb2d2e269d0abb1` Hash scope: Hash scope not separately documented; inspect source record
Diagram steps ["Fix reference profile and taxonomy","Generate candidate abundance profiles","Align taxa and ranks","Score detection and abundance separately"] Individual claims	CAMI-challenge/OPAL official source Original source ↗ README: Overview, Computed metrics and Inputs Version: 98120c326eef08e391899e4bd3a362e0e6558b4a Retrieved: 2026-09-16T10:30:23.429372+00:00	source checked automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.diagram.steps` Source artifact SHA-256: `d123b202234d3ed520911982dcf88e2fcfd888c83e65378effb2d2e269d0abb1` Hash scope: Hash scope not separately documented; inspect source record
Diagram title Conceptual evaluation workflow Individual claims	CAMI-challenge/OPAL official source Original source ↗ README: Overview, Computed metrics and Inputs Version: 98120c326eef08e391899e4bd3a362e0e6558b4a Retrieved: 2026-09-16T10:30:23.429372+00:00	source checked automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.diagram.title` Source artifact SHA-256: `d123b202234d3ed520911982dcf88e2fcfd888c83e65378effb2d2e269d0abb1` Hash scope: Hash scope not separately documented; inspect source record
Entity type Task guide; concrete protocol identities remain separate. Individual claims	CAMI-challenge/OPAL official source Original source ↗ README: Overview, Computed metrics and Inputs Version: 98120c326eef08e391899e4bd3a362e0e6558b4a Retrieved: 2026-09-16T10:30:23.429372+00:00	source checked automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.facts.0.value` Source artifact SHA-256: `d123b202234d3ed520911982dcf88e2fcfd888c83e65378effb2d2e269d0abb1` Hash scope: Hash scope not separately documented; inspect source record
Datasets No single dataset is fixed by this guide. Select a linked protocol and its versioned data release. Individual claims	CAMI-challenge/OPAL official source Original source ↗ README: Overview, Computed metrics and Inputs Version: 98120c326eef08e391899e4bd3a362e0e6558b4a Retrieved: 2026-09-16T10:30:23.429372+00:00	inapplicable automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.facts.1.value` Source artifact SHA-256: `d123b202234d3ed520911982dcf88e2fcfd888c83e65378effb2d2e269d0abb1` Hash scope: Hash scope not separately documented; inspect source record
Organisms No shared organism population is defined at this guide level. Record it for each selected dataset. Individual claims	CAMI-challenge/OPAL official source Original source ↗ README: Overview, Computed metrics and Inputs Version: 98120c326eef08e391899e4bd3a362e0e6558b4a Retrieved: 2026-09-16T10:30:23.429372+00:00	inapplicable automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.facts.2.value` Source artifact SHA-256: `d123b202234d3ed520911982dcf88e2fcfd888c83e65378effb2d2e269d0abb1` Hash scope: Hash scope not separately documented; inspect source record
Assays No single measurement assay is fixed by this guide; the endpoint and assay belong to the selected protocol. Individual claims	CAMI-challenge/OPAL official source Original source ↗ README: Overview, Computed metrics and Inputs Version: 98120c326eef08e391899e4bd3a362e0e6558b4a Retrieved: 2026-09-16T10:30:23.429372+00:00	inapplicable automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.facts.3.value` Source artifact SHA-256: `d123b202234d3ed520911982dcf88e2fcfd888c83e65378effb2d2e269d0abb1` Hash scope: Hash scope not separately documented; inspect source record
Splits No executable split is attached to this task identity. Use the selected protocol’s split manifest. Individual claims	CAMI-challenge/OPAL official source Original source ↗ README: Overview, Computed metrics and Inputs Version: 98120c326eef08e391899e4bd3a362e0e6558b4a Retrieved: 2026-09-16T10:30:23.429372+00:00	inapplicable automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.facts.4.value` Source artifact SHA-256: `d123b202234d3ed520911982dcf88e2fcfd888c83e65378effb2d2e269d0abb1` Hash scope: Hash scope not separately documented; inspect source record
Allowed inputs Predicted and reference taxonomic profiles with consistent taxon identifiers and abundance definitions. Individual claims	CAMI-challenge/OPAL official source Original source ↗ README: Overview, Computed metrics and Inputs Version: 98120c326eef08e391899e4bd3a362e0e6558b4a Retrieved: 2026-09-16T10:30:23.429372+00:00	source checked automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.facts.5.value` Source artifact SHA-256: `d123b202234d3ed520911982dcf88e2fcfd888c83e65378effb2d2e269d0abb1` Hash scope: Hash scope not separately documented; inspect source record
Adaptation No common fitting regime is imposed here. Keep pretrained, frozen, probed, fine-tuned and conventional methods distinct where applicable. Individual claims	CAMI-challenge/OPAL official source Original source ↗ README: Overview, Computed metrics and Inputs Version: 98120c326eef08e391899e4bd3a362e0e6558b4a Retrieved: 2026-09-16T10:30:23.429372+00:00	inapplicable automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.facts.6.value` Source artifact SHA-256: `d123b202234d3ed520911982dcf88e2fcfd888c83e65378effb2d2e269d0abb1` Hash scope: Hash scope not separately documented; inspect source record

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

Release 2026-09-17-d277315f7d76 · Record review: discovered

4 source records and release history

MetaPhlAn official resource · Original source · Not reported
Critical Assessment of Metagenome Interpretation: the second round of challenges · Original source · PMC9007738
Assessing taxonomic metagenome profilers with OPAL · Original source · PMC6398228
CAMI-challenge/OPAL official source · Original source · 98120c326eef08e391899e4bd3a362e0e6558b4a

Download this release

Technical metadata and extraction receipts

Stable ID: catalog-task-community-profiling

areas: microbes-communities
entity level: task
version: Not reported
task: Community profiling
scope note: Estimate taxon abundances in metagenomic samples.
benchmark research: review date: 2026-09-17; status: broad_task_requires_concrete_protocol_links; primary sources: evidence-expansion-p2-evidence-discovery-final-cami2-da932c1cde8b; evidence-expansion-p2-evidence-discovery-final-opal-a4503ef32e99; inspected locators: Table1 rankings; main figures and linked supplementary material; Methods and benchmark Results; figure captions and source availability; searched queries: Community profiling primary paper benchmark results; gaps: OPAL is an evaluator rather than a single dataset. Numerical comparisons require selected community dataset, taxonomic rank and profiling output version. No complete main-text score matrix; source data remains pending.; claim scope: Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.
historical missing metadata: protocol version: not_yet_extracted
metadata review scope: historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
legacy kinds: benchmark
entity classification: review date: 2026-09-17; rationale: This record identifies the biological prediction question or a suite-specific task, rather than a uniquely fixed evaluated procedure. Preserve its task identity and leave split, model adaptation and scoring details on linked protocols/evaluations.; source ids: src-discovery-cami-challenge-opal; source locator: README: Overview, Computed metrics and Inputs; ambiguities: None recorded

Related records

Suggest a correction