Validation and assessment of variant calling pipelines for next-generation sequencing

Abstract

BackgroundThe processing and analysis of the large scale data generated by next-generation sequencing (NGS) experiments is challenging and is a burgeoning area of new methods development. Several new bioinformatics tools have been developed for calling sequence variants from NGS data. Here, we validate the variant calling of these tools and compare their relative accuracy to determine which data processing pipeline is optimal.ResultsWe developed a unified pipeline for processing NGS data that encompasses four modules: mapping, filtering, realignment and recalibration, and variant calling. We processed 130 subjects from an ongoing whole exome sequencing study through this pipeline. To evaluate the accuracy of each module, we conducted a series of comparisons between the single nucleotide variant (SNV) calls from the NGS data and either gold-standard Sanger sequencing on a total of 700 variants or array genotyping data on a total of 9,935 single-nucleotide polymorphisms. A head to head comparison showed that Genome Analysis Toolkit (GATK) provided more accurate calls than SAMtools (positive predictive value of 92.55% vs. 80.35%, respectively). Realignment of mapped reads and recalibration of base quality scores before SNV calling proved to be crucial to accurate variant calling. GATK HaplotypeCaller algorithm for variant calling outperformed the UnifiedGenotype algorithm. We also showed a relationship between mapping quality, read depth and allele balance, and SNV call accuracy. However, if best practices are used in data processing, then additional filtering based on these metrics provides little gains and accuracies of >99% are achievable.ConclusionsOur findings will help to determine the best approach for processing NGS data to confidently call variants for downstream analyses. To enable others to implement and replicate our results, all of our codes are freely available at http://metamoodics.org/wes.

Scaling accurate genetic variant discovery to tens of thousands of samples

“…In our results, the HaplotypeCaller was the most sensitive indel variant caller using both whole genome and exome-capture data, which is a conclusion consistent with other reports comparing variant callers [22,21,9]. However, in contrast with the report of Rimmer, et al, the results here show that the HaplotypeCaller had the lowest FDR on whole genomic data, suggesting that it had the highest specificity for indel variant calls [22].…”

From next-generation resequencing reads to a high-quality variant data set

“…In fact, no single method captures all existing variation and many tools provide contrasting and complementary information (Liu et al, 2013a;Neuman et al, 2013;O'Rawe et al, 2013;Yu and Sun, 2013;Cheng et al, 2014;Pirooznia et al, 2014). As a consequence, it is recommended to combine independent data sets from multiple methods to achieve better specificity or sensitivity.…”

A reference data set of 5.4 million phased human variants validated by genetic inheritance from sequencing a three-generation 17-member pedigree

“…One approach is to test variant calls made by an NGS method using an orthogonal technology (e.g., array-based genotyping or Sanger sequencing) and then to measure the degree of concordance between results (Ajay et al 2011; The 1000 Genomes Project Consortium 2012; Pirooznia et al 2014). This approach can provide a measure of precision of a variant caller, but not recall, as recall estimates require knowledge of what is missed.…”