rewire.it
Protocol

Species profiling on Dilthey2019 simulated long reads (Long-read taxonomic profiling)

Species profiling on Dilthey2019 simulated long reads · Table 1:. Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.

Lightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:: Recall, Species profiling on Dilthey2019 simulated long reads

7 evaluations · 21 metric rows

At a glance

Explanatory profile: limited source coverage · Automated source review, 2026-09-17. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.

How it works

Evaluation in this paper

Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.

Lightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:: Recall, Species profiling on Dilthey2019 simulated long reads

Evaluation design

Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.

These source-backed links do not make different protocols or scores interchangeable.

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Published comparisons

Explore the results reported under one evaluation protocol. Each figure keeps its source, dataset and metric together; it is not a ranking across studies.

Species profiling on Dilthey2019 simulated long reads · Table 1:

Recall (fraction) · Higher values are better for this metric.

Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.

Evaluation protocol · Species profiling on Dilthey2019 simulated long reads

  1. Lemur · Configuration · Author-reported evaluation0.951
  2. Lemur + Magnet · Configuration · Author-reported evaluation0.927
  3. Melon · Configuration · Independent external evaluation0.963
  4. MetaMaps · Configuration · Result quoted from another source1.000
  5. Sourmash · Configuration · Independent external evaluation0.927
  6. Centrifuger · Configuration · Independent external evaluation0.774
  7. Kraken 2 · Configuration · Independent external evaluation0.976

Source order is preserved. Plotted marks show point estimates; uncertainty, where reported, is retained in the printed values and table. Differences do not establish statistical significance.

Lightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:: Recall, Species profiling on Dilthey2019 simulated long reads
Values, uncertainty and evidence
Recall: original source values
Tested entityPrinted valueUncertaintyEvidence
Lemur · Configuration0.951 fractionNot reportedAuthor-reported evaluation · source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Lemur, column Recall; XML row2 column2
Lemur + Magnet · Configuration0.927 fractionNot reportedAuthor-reported evaluation · source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Lemur + Magnet, column Recall; XML row3 column2
Melon · Configuration0.963 fractionNot reportedIndependent external evaluation · source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Melon, column Recall; XML row4 column2
MetaMaps · Configuration1.000 fractionNot reportedResult quoted from another source · source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row MetaMaps, column Recall; XML row5 column2
Sourmash · Configuration0.927 fractionNot reportedIndependent external evaluation · source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Sourmash, column Recall; XML row6 column2
Centrifuger · Configuration0.774 fractionNot reportedIndependent external evaluation · source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Centrifuger, column Recall; XML row7 column2
Kraken 2 · Configuration0.976 fractionNot reportedIndependent external evaluation · source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Kraken 2, column Recall; XML row8 column2
Scope and limitations
  • MetaMaps is quoted prior evidence, not an independent repeated run.
  • Reference databases and classifier-versus-profiler task differ.
  • No interval assigned unless printed in source cell.

Source transcription and grouping reviewed by automated source review on 2026-09-17. These experiments were not independently reproduced by rewire.

Tested entities and results

Release 2026-09-17-d277315f7d76 · 7 evaluations · 21 metric rows. Different protocols are not a single leaderboard.

Results grouped by the exact reported evaluation
Metric and findingCoverage and uncertaintyEvidence
Sourmash: Species profiling on Dilthey2019 simulated long reads

Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.

Independent external evaluation · Evaluation metadata: needs review

0.932 F1 score

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Sourmash, column F1 score; XML row6 column4

Source checking is not independent reproduction.

0.927 Recall

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Sourmash, column Recall; XML row6 column2

Source checking is not independent reproduction.

0.938 Precision

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Sourmash, column Precision; XML row6 column3

Source checking is not independent reproduction.

Lemur + Magnet: Species profiling on Dilthey2019 simulated long reads

Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.

Author-reported evaluation · Evaluation metadata: needs review

0.950 Precision

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Lemur + Magnet, column Precision; XML row3 column3

Source checking is not independent reproduction.

0.938 F1 score

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Lemur + Magnet, column F1 score; XML row3 column4

Source checking is not independent reproduction.

0.927 Recall

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Lemur + Magnet, column Recall; XML row3 column2

Source checking is not independent reproduction.

Kraken 2: Species profiling on Dilthey2019 simulated long reads

Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.

Independent external evaluation · Evaluation metadata: needs review

0.055 Precision

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Kraken 2, column Precision; XML row8 column3

Source checking is not independent reproduction.

0.976 Recall

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Kraken 2, column Recall; XML row8 column2

Source checking is not independent reproduction.

0.104 F1 score

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Kraken 2, column F1 score; XML row8 column4

Source checking is not independent reproduction.

Lemur: Species profiling on Dilthey2019 simulated long reads

Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.

Author-reported evaluation · Evaluation metadata: needs review

0.703 Precision

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Lemur, column Precision; XML row2 column3

Source checking is not independent reproduction.

0.808 F1 score

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Lemur, column F1 score; XML row2 column4

Source checking is not independent reproduction.

0.951 Recall

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Lemur, column Recall; XML row2 column2

Source checking is not independent reproduction.

Melon: Species profiling on Dilthey2019 simulated long reads

Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.

Independent external evaluation · Evaluation metadata: needs review

0.946 F1 score

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Melon, column F1 score; XML row4 column4

Source checking is not independent reproduction.

0.929 Precision

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Melon, column Precision; XML row4 column3

Source checking is not independent reproduction.

0.963 Recall

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Melon, column Recall; XML row4 column2

Source checking is not independent reproduction.

Centrifuger: Species profiling on Dilthey2019 simulated long reads

Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.

Independent external evaluation · Evaluation metadata: needs review

0.774 Recall

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Centrifuger, column Recall; XML row7 column2

Source checking is not independent reproduction.

0.050 Precision

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Centrifuger, column Precision; XML row7 column3

Source checking is not independent reproduction.

0.093 F1 score

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row Centrifuger, column F1 score; XML row7 column4

Source checking is not independent reproduction.

MetaMaps: Species profiling on Dilthey2019 simulated long reads

Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.

Result quoted from another source · Evaluation metadata: needs review

0.862 Precision

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row MetaMaps, column Precision; XML row5 column3

Source checking is not independent reproduction.

1.000 Recall

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row MetaMaps, column Recall; XML row5 column2

Source checking is not independent reproduction.

0.926 F1 score

Unit: fraction · Direction: higher

Uncertainty: unreported

Scored: Not reported · Eligible: Not reported

source checkedLightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet · Table 1:, row MetaMaps, column F1 score; XML row5 column4

Source checking is not independent reproduction.

Papers and result coverage

Last literature check: 2026-09-17. Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.

Paper or primary resourceVersionReference
Lightweight taxonomic profiling of long-read metagenomic datasets with Lemur and MagnetPMC archival version PMC11185576.2Read source
DOI: 10.1101/2024.06.01.596961

What is still missing

  • Independent batch review before import; preserve existing observation identities.
Search and extraction details

complete comparison tables extracted pending publication review

Searches

  • Lightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet primary paper benchmark results

Evidence locations

  • Table 1:: Recall, Species profiling on Dilthey2019 simulated long reads

Strengths and limitations

Strengths and considerations

No source-reviewed explanatory claims are recorded here yet.

Limitations and conditions

No source-reviewed explanatory claims are recorded here yet.

Profile review details

Primary-source transcription and separate automated review. No human sign-off or experimental reproduction.

Stable record: paper-protocol-1080592b8d6a8ce0bd

Evidence table

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

3 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-17-d277315f7d76
Property and statementOriginal source and locationReview and provenance
Evaluation in this paper

Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.

Individual claims
Lightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet

Original source ↗

Table 1:: Recall, Species profiling on Dilthey2019 simulated long reads

Version: PMC archival version PMC11185576.2
Retrieved: 2026-09-17T07:56:16.880151+00:00

source checked

automated source review · 2026-09-17

Audit details

Primary-source transcription and separate automated review. No human sign-off or experimental reproduction.

Field: attributes.profile.sections.0.body

Source artifact SHA-256: 4afb9195da447916eb6f733816e3640741c7ade08ea8920d205c3be7b3cce27a

Hash scope: Exact retrieved primary paper artifact bytes.

Inspected artifact

Introduction

Species profiling on Dilthey2019 simulated long reads · Table 1:. Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.

Individual claims
Lightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet

Original source ↗

Table 1:: Recall, Species profiling on Dilthey2019 simulated long reads

Version: PMC archival version PMC11185576.2
Retrieved: 2026-09-17T07:56:16.880151+00:00

source checked

automated source review · 2026-09-17

Audit details

Primary-source transcription and separate automated review. No human sign-off or experimental reproduction.

Field: attributes.profile.summary

Source artifact SHA-256: 4afb9195da447916eb6f733816e3640741c7ade08ea8920d205c3be7b3cce27a

Hash scope: Exact retrieved primary paper artifact bytes.

Inspected artifact

Relationship: evaluates task

reported-task-6330d593980b5b

Individual claims
Lightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet

Original source ↗

Table 1:: Recall, Species profiling on Dilthey2019 simulated long reads

Version: PMC archival version PMC11185576.2
Retrieved: 2026-09-17T07:56:16.880151+00:00

source checked

automated source review · 2026-09-17

Audit details

Field: links:evaluates_task:reported-task-6330d593980b5b

Claim: paper-claim-4e9825e29807055d9c

Source artifact SHA-256: 4afb9195da447916eb6f733816e3640741c7ade08ea8920d205c3be7b3cce27a

Hash scope: Exact retrieved primary paper artifact bytes.

Inspected artifact

Sources and history

Release 2026-09-17-d277315f7d76 · Record review: needs review

Download this release
Technical metadata and extraction receipts

Stable ID: paper-protocol-1080592b8d6a8ce0bd

areas
microbes-communities
tasks
Long-read taxonomic profiling
entity level
protocol
protocol
Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.
comparison panels
id: part2-lemur-magnet-2024-T1-a3172c2928; title: Species profiling on Dilthey2019 simulated long reads · Table 1:; protocol id: paper-protocol-1080592b8d6a8ce0bd; dataset id: paper-dataset-e8eb310c5d40e647ac; metric: Recall; unit: fraction; direction: higher; result ids: paper-result-80f8cca2c2d9379a1a; paper-result-620e5294f3d001ac96; paper-result-a91cd2aadeccebf9b9; paper-result-b70bf4b4c41e6dd812; paper-result-6da2d59412e925bccd; paper-result-74d2e70b442ba9026b; paper-result-8fd5837795f2fbba5f; source ids: part2-lemur-magnet-2024; source locator: Table 1:: Recall, Species profiling on Dilthey2019 simulated long reads; context: Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.; caveats: MetaMaps is quoted prior evidence, not an independent repeated run.; Reference databases and classifier-versus-profiler task differ.; No interval assigned unless printed in source cell.; review: method: automated_source_review; date: 2026-09-17; id: part2-lemur-magnet-2024-T1-81ab2d846d; title: Species profiling on Dilthey2019 simulated long reads · Table 1:; protocol id: paper-protocol-1080592b8d6a8ce0bd; dataset id: paper-dataset-e8eb310c5d40e647ac; metric: Precision; unit: fraction; direction: higher; result ids: paper-result-2268ad4341b9316164; paper-result-06a95cb8e4f61822eb; paper-result-9a39bac49d7dcf2b79; paper-result-a6256bb040cd29f64f; paper-result-f53c61bd32688408d8; paper-result-e2ceeb0f245677c087; paper-result-0dc829636e77fc0d92; source ids: part2-lemur-magnet-2024; source locator: Table 1:: Precision, Species profiling on Dilthey2019 simulated long reads; context: Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.; caveats: MetaMaps is quoted prior evidence, not an independent repeated run.; Reference databases and classifier-versus-profiler task differ.; No interval assigned unless printed in source cell.; review: method: automated_source_review; date: 2026-09-17; id: part2-lemur-magnet-2024-T1-e9572b0ed9; title: Species profiling on Dilthey2019 simulated long reads · Table 1:; protocol id: paper-protocol-1080592b8d6a8ce0bd; dataset id: paper-dataset-e8eb310c5d40e647ac; metric: F1 score; unit: fraction; direction: higher; result ids: paper-result-4468fa08040fb0bd80; paper-result-19f79487e8dff1a781; paper-result-25455d4621f0aeb383; paper-result-f9b8d072704477374e; paper-result-03860ee74178c46356; paper-result-fdc2db2e65a30e98e7; paper-result-f8adf74d2b525a6380; source ids: part2-lemur-magnet-2024; source locator: Table 1:: F1 score, Species profiling on Dilthey2019 simulated long reads; context: Species-level recall/precision/F 1. MetaMaps results copied from original manuscript; other tool versions reported in Methods. 200,114 reads;96 strain design,94 strains with RefSeq species representatives; average read 4,997 bp.; caveats: MetaMaps is quoted prior evidence, not an independent repeated run.; Reference databases and classifier-versus-profiler task differ.; No interval assigned unless printed in source cell.; review: method: automated_source_review; date: 2026-09-17
benchmark research
review date: 2026-09-17; status: complete_comparison_tables_extracted_pending_publication_review; primary sources: part2-lemur-magnet-2024; inspected locators: Table 1:: Recall, Species profiling on Dilthey2019 simulated long reads; searched queries: Lightweight taxonomic profiling of long-read metagenomic datasets with Lemur and Magnet primary paper benchmark results; gaps: Independent batch review before import; preserve existing observation identities.; claim scope: Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.
legacy kinds
benchmark
entity classification
review date: 2026-09-17; rationale: The source-backed record identifies a specified evaluated procedure and its dataset/split/scoring context. Classify it as a protocol while preserving version and comparison restrictions.; source ids: part2-lemur-magnet-2024; source locator: Table 1:: Recall, Species profiling on Dilthey2019 simulated long reads; ambiguities: None recorded
Related records

Suggest a correction