From defaults to databases: parameter and database choice dramatically impact the performance of metagenomic taxonomic classification tools

Abstract

In metagenomic analyses of microbiomes, one of the first steps is usually the taxonomic classification of reads by comparison to a database of previously taxonomically classified genomes. While different studies comparing metagenomic taxonomic classification methods have determined that different tools are ‘best’, there are two tools that have been used the most to-date: Kraken (k-mer-based classification against a user-constructed database) and MetaPhlAn (classification by alignment to clade-specific marker genes), the latest versions of which are Kraken2 and MetaPhlAn 3, respectively. We found large discrepancies in both the proportion of reads that were classified as well as the number of species that were identified when we used both Kraken2 and MetaPhlAn 3 to classify reads within metagenomes from human-associated or environmental datasets. We then investigated which of these tools would give classifications closest to the real composition of metagenomic samples using a range of simulated and mock samples and examined the combined impact of tool–parameter–database choice on the taxonomic classifications given. This revealed that there may not be a one-size-fits-all ‘best’ choice. While Kraken2 can achieve better overall performance, with higher precision, recall and F1 scores, as well as alpha- and beta-diversity measures closer to the known composition than MetaPhlAn 3, the computational resources required for this may be prohibitive for many researchers, and the default database and parameters should not be used. We therefore conclude that the best tool–parameter–database choice for a particular application depends on the scientific question of interest, which performance metric is most important for this question and the limit of available computational resources.

A novel barcoded nanopore sequencing workflow of high-quality, full-length bacterial 16S amplicons for taxonomic annotation of bacterial isolates and complex microbial communities

“…Given that many studies and microbiome-based diagnostics prioritize composite metrics (e.g., SD) (27–29), both 16S-based approaches tested in our workflow offer a suitable and cost-effective alternative for characterizing alpha diversity metrics. In terms of inter-sample diversity, and in agreement with previously published studies (30), we observed disparities in taxonomic profiling among the various sequencing technologies (31, 32) and bioinformatics pipelines (33, 34), but the magnitude of these differences remained low (15.4% and 13.9% variability for primers A and B, respectively). While the reasons for differences observed between 16S-based and shotgun-based profiling have been described elsewhere (30), a significant proportion of this variability can be explained with the inability to perfectly match the taxonomies derived from the different analytical pipelines used in this study.…”

Enhanced tameness byLimosilactobacillus reuterifrom gut microbiota of selectively bred mice

“…For taxonomic analysis, to capture the full diversity of WHS mice gut microbiome, we used the full NCBI/RefSeq prokaryote genome sequence database (NCBI RefSeq Complete V205) (24).…”

“…0. (1189 GB)) (24) to ensure comprehensive species identification within the gut samples. Default parameters were used for Kraken 2, with a confidence score threshold set at 0.15.…”

nf-core/taxprofiler: highly parallelised and flexible pipeline for metagenomic taxonomic classification and profiling

“…Consequently, the size of reference databases of taxonomic classifiers is also growing, often outpacing the computational capacity available to researchers. In fact, while this was one of the main motivations behind classifiers such as Kraken2 (Wood, Lu, and Langmead 2019), these algorithmic techniques are already becoming insufficient (Wright, Comeau, and Langille 2023).…”