Systematic benchmarking of omics computational tools

Mangul, Serghei; Martin, Lana S.; Hill, Brian; Lam, Angela Ka-Mei; Distler, Margaret G.; Zelikovsky, Alexander; Eskin, Eleazar; Flint, Jonathan

doi:10.1038/s41467-019-09406-4

Cited by 121 publications

(124 citation statements)

References 62 publications

Supporting

Mentioning

121

Contrasting

Order By: Relevance

“…Differences between the reads and the reference genome were consider as errors. Since this approach is unable to differentiate between the real errors and single nucleotide variants it oversimplifies the error correction problem and may bias benchmarking results 12 . In addition, error correction algorithms have undergone significant development since the prior benchmarking studies, and the performance of the newest methods has not yet been evaluated.…”

Section: Introductionmentioning

confidence: 99%

Benchmarking of computational error-correction methods for next-generation sequencing data

Mitchell

Mandric

Martin

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Recent advancements in next-generation sequencing have rapidly improved our ability to study genomic material at an unprecedented scale. Despite substantial improvements in sequencing technologies, the errors present in the data still risk confounding downstream analysis and limiting the applicability of sequencing technologies in clinical tools. Computational error-correction promises to eliminate sequencing errors, but the relative performance of error correction algorithms remains unknown. In this paper, we evaluate the ability of error-correction algorithms to fix errors across different types of datasets that contain various level of heterogeneity. We highlight the advantages and limitations of computational error correction techniques across different domains of biology, including immunogenomics and virology. To demonstrate the efficacy of our technique, w e applied the UMI-based high-fidelity sequencing protocol to eliminate sequencing errors from both simulated data and the raw reads. We then performed a realistic evaluation of error correction methods. We find that method performance varies substantially across different types of datasets, with no single method performing best on all types of examined data. Finally, we also identified the techniques that offer a good balance between precision and sensitivity.

show abstract

Section: Introductionmentioning

confidence: 99%

Benchmarking of computational error-correction methods for next-generation sequencing data

Mitchell

Mandric

Martin

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…There is now a growing body of evidence that systematic benchmarking studies stimulate research communities, help to set operating standards for evaluating computational models and methods, and lower the barriers for introducing new ideas in the field (36)(37)(38) .…”

Section: Discussionmentioning

confidence: 99%

Benchmarking kinetic models ofEscherichia colimetabolism

Shepelin

Machado

Nielsen

et al. 2020

Preprint

View full text Add to dashboard Cite

Predicting phenotype from genotype is the holy grail of quantitative systems biology.Kinetic models of metabolism are among the most mechanistically detailed tools for phenotype prediction. Kinetic models describe changes in metabolite concentrations as a function of enzyme concentration, reaction rates, and concentrations of metabolic effectors uniquely enabling integration of multiple omics data types in a unifying mechanistic framework. While development of such models for Escherichia coli has been going on for almost twenty years, multiple separate models have been established and systematic independent benchmarking studies have not been performed on the full set of models available. In this study we compared systematically all recently published kinetic models of the central carbon metabolism of Escherichia coli . We assess the ease of use of the models, their ability to include omics data as input, and the accuracy of prediction of central carbon metabolic flux phenotypes. We conclude that there is no clear winner among the models when considering the resulting tradeoffs in performance

show abstract

“…Conversely, NeuSomatic, which uses a previously trained neural network for variant detection, worked quite well. An issue of general concern in benchmarking studies is whether results from simulated data generalize to empirical data sets [20]. Whenever ground truth information is required for a multitude of data sets, simulations are a useful tool.…”

Section: Discussionmentioning

confidence: 99%

“…To our knowledge, an independent assessment of variant callers using M-seq tumor data has never been carried out. Indeed, third-party benchmarking is important because authors evaluating their own methods against others may be prone to the "self-assessment trap" [20], i.e., implicit biases in the evaluation conditions. Besides, previous comparisons have explored a limited range of rather simple multiregional scenarios.…”

Section: Introductionmentioning

confidence: 99%

Accuracy of somatic variant detection in multiregional tumor sequencing data

Detering

Tomás

Prieto

et al. 2019

Preprint

View full text Add to dashboard Cite

Multiregional bulk sequencing data is necessary to characterize intratumor genetic heterogeneity. Novel somatic variant calling approaches aim to address the particular characteristics of multiregional data, but it remains unclear to which extent they improve compared to singlesample strategies. Here we compared the performance of 16 single-nucleotide variant calling approaches on multiregional sequencing data under different scenarios with in-silico and real sequencing reads, including varying sequencing coverage and increasing levels of spatial clonal admixture. Under the conditions simulated, methods that use information across multiple samples do not necessarily perform better than some of the standard calling methods that work sample by sample. Nonetheless, our results indicate that under difficult conditions, Mutect2 in multisample mode, in combination with a correction step, seems to perform best. Our analysis provides data-driven guidance for users and developers of somatic variant calling tools.

show abstract

Systematic benchmarking of omics computational tools

Cited by 121 publications

References 62 publications

Benchmarking of computational error-correction methods for next-generation sequencing data

Benchmarking of computational error-correction methods for next-generation sequencing data

Benchmarking kinetic models ofEscherichia colimetabolism

Accuracy of somatic variant detection in multiregional tumor sequencing data

Contact Info

Product

Resources

About