Comparison of GATK and DeepVariant by trio sequencing

Abstract

While next-generation sequencing (NGS) has transformed genetic testing, it generates large quantities of noisy data that require a significant amount of bioinformatics to generate useful interpretation. The accuracy of variant calling is therefore critical. Although GATK HaplotypeCaller is a widely used tool for this purpose, newer methods such as DeepVariant have shown higher accuracy in assessments of gold-standard samples for whole-genome sequencing (WGS) and whole-exome sequencing (WES), but a side-by-side comparison on clinical samples has not been performed. Trio WES was used to compare GATK (4.1.2.0) HaplotypeCaller and DeepVariant (v0.8.0). The performance of the two pipelines was evaluated according to the Mendelian error rate, transition-to-transversion (Ti/Tv) ratio, concordance rate, and pathological variant detection rate. Data from 80 trios were analyzed. The Mendelian error rate of the 77 biological trios calculated from the data by DeepVariant (3.09 ± 0.83%) was lower than that calculated from the data by GATK (5.25 ± 0.91%) (p < 0.001). DeepVariant also yielded a higher Ti/Tv ratio (2.38 ± 0.02) than GATK (2.04 ± 0.07) (p < 0.001), suggesting that DeepVariant proportionally called more true positives. The concordance rate between the 2 pipelines was 88.73%. Sixty-three disease-causing variants were detected in the 80 trios. Among them, DeepVariant detected 62 variants, and GATK detected 61 variants. The one variant called by DeepVariant but not GATK HaplotypeCaller might have been missed by GATK HaplotypeCaller due to low coverage. OTC exon 2 (139 bp) deletion was not detected by either method. Mendelian error rate calculation is an effective way to evaluate variant callers. By this method, DeepVariant outperformed GATK, while the two pipelines performed equally in other parameters.

Comparison of calling pipelines for whole genome sequencing: an empirical study demonstrating the importance of mapping and alignment

“…The aspect of accuracy is not only demonstrated by the repeated sequencing of the GIAB sample HG002 and its comparison with the truth set data, but also by the Mendelian inheritance errors detected in the repeatedly sequenced GIAB trio. On the one hand, DeepVariant had fewer Mendel errors than GATK-based variant calling on parent-offspring trios, which is in line with the findings of Lin and colleagues 6 . However, when the DRAGEN was used for mapping and alignment, there was almost no difference between DeepVariant and GATK in the variant calling step.…”

“…The winner of the second precisionFDA Truth Challenge for short-read sequencing in 2020 was Illumina's DRAGEN 8 . It outperformed other pipelines 6,[9][10][11][12] , in particular in difficult-to-call genomic regions 13 . It is, however, unclear which pipeline should be used for secondary analysis of WGS data, i.e., from fastq to vcf, in terms of both speed and accuracy.…”

“…It is, however, unclear which pipeline should be used for secondary analysis of WGS data, i.e., from fastq to vcf, in terms of both speed and accuracy. The best choice is challenging because recent pipeline comparisons relied on synthetic-diploid and simulated data 14 or did not include the award-winning DeepVariant and/or DRAGEN pipelines 6,9,10,12,15 . Furthermore, several comparisons focused only on variant calling, the last step in the secondary analysis pipeline.…”

Species-wide genomics of kākāpō provides transformational tools to accelerate recovery

“…Reference genomes and associated genomic resources function as living documents, with each new study or analysis providing an opportunity to update the genome, its associated annotation, and the genomic data for each individual [26][27][28][29] . For example, new variant calling methods, especially those that incorporate machine learning models, can be used for increased accuracy [30][31][32][33][34] .…”

Towards increased accuracy and reproducibility in SARS-CoV-2 next generation sequence analysis for public health surveillance

“…Reference alignment-based variant discovery using NGS data is not new, and has been widely used in human population studies. The performance of such approaches using different processing pipelines, including read aligners, variant callers, and sequencing instruments, has been assessed by various studies [3][4][5][6][7][8][9] resulting in best practices for germline 10 and somatic variant detections.…”