Best practices for evaluating single nucleotide variant calling methods for microbial genomics

Abstract

Innovations in sequencing technologies have allowed biologists to make incredible advances in understanding biological systems. As experience grows, researchers increasingly recognize that analyzing the wealth of data provided by these new sequencing platforms requires careful attention to detail for robust results. Thus far, much of the scientific Communit’s focus for use in bacterial genomics has been on evaluating genome assembly algorithms and rigorously validating assembly program performance. Missing, however, is a focus on critical evaluation of variant callers for these genomes. Variant calling is essential for comparative genomics as it yields insights into nucleotide-level organismal differences. Variant calling is a multistep process with a host of potential error sources that may lead to incorrect variant calls. Identifying and resolving these incorrect calls is critical for bacterial genomics to advance. The goal of this review is to provide guidance on validating algorithms and pipelines used in variant calling for bacterial genomics. First, we will provide an overview of the variant calling procedures and the potential sources of error associated with the methods. We will then identify appropriate datasets for use in evaluating algorithms and describe statistical methods for evaluating algorithm performance. As variant calling moves from basic research to the applied setting, standardized methods for performance evaluation and reporting are required; it is our hope that this review provides the groundwork for the development of these standards.

Assuring the Quality of Next-Generation Sequencing in Clinical Microbiology and Public Health Laboratories

“…To address these needs, the National Institute of Standards and Technology (NIST) is currently working to develop RMs for bacterial genomic sequencing. The strains chosen are relevant to food safety and clinical microbiology NGS applications and represent diverse genome sizes, plasmid contents, and GC contents (10). Other RMs that may be useful for NGS for validation and evaluation of the bioinformatics pipeline include synthetic DNA samples, such as plasmids containing known variants that can be engineered to represent a broad range of sequences and variant types or synthetic "armored RNA" capsids that can be used to simulate infectious viral particles spiked into clinical matrices; and electronic or digital reference materials, such as curated benchmark data sets (i.e., well-characterized and complete genome data sets derived from a variety of organisms relevant to clinical and public health microbiology) (10,11,16,30).…”

“…Additional issues have been identified that should be addressed to fully realize the integration of NGS into the clinical and public health laboratory setting. These include reducing the cost and turnaround time of sequencing, the development of fully automated user-friendly sequencing and data analysis pipelines, the creation of comprehensive and well-curated reference genome databases, curation of genotype-phenotype correlations for clinically relevant microorganisms (for example, when making predictions about antimicrobial resistance), establishment of proficiency testing (PT) and quality control (QC) measures, and the development of practice guidelines to ensure the quality of NGS-based tests (7,(9)(10)(11)(12). This review highlights currently available standards and guidelines for the implementation of NGS in the clinical and public health laboratory setting, and it includes considerations for NGS test validation, QC procedures, PT, and reference materials.…”

PEPR: pipelines for evaluating prokaryotic references

“…To increase confidence in WGS assay, results a critical assessment of the errors inherent to the measurement processes is required. A number of sources of error associated with the WGS measurement process have been identified, but the degree to which they can be predicted, controlled, or compensated varies significantly [1].…”

Genomic studies of biochemical compounds determining arabica coffee (Coffea arabica L.) quality

“…High sequencing depth is a key consideration in genomic analysis for polyploidy like arabica coffee. Generally, for tetraploids, a sequencing depth greater than 20x, as obtained from bulk sequencing in this study, would have minimal impact of random polymerase errors on variant calls (sumarised by Olson et al, 2015).…”

“…Among several sequencing technologies, Illumina paired-end sequencing was used in this study due to its affordability and ability to produce the high read depth required for unambiguously detecting alleles in a polyploidy, and for the improvement of read mapping and assembly accuracy (Clevenger et al, 2015). The use of paired-end library preparation methodology can also reduce variant calling error as it reduces duplicate mapping errors (Olson et al, 2015;Schlötterer et al, 2014). These authors suggested longer reads (paired-end 150 or greater) were suitable for species with highly similar subgenomes (e.g.…”