Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays

Krishnan, Vandhana; Utiramerur, Sowmi; Ng, Zena; Datta, Somalee; Snyder, M; Ashley, Euan A.

doi:10.1101/643163

Cited by 9 publications

(13 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is a necessity of bioinformatic pipelines for variant calling analysis on WGS data in a precise and efficient way prior to their integration into clinical diagnostic applications 14,15 . In general, a pipeline is comprised of the following steps: quality control, read alignment, variant calling, annotation, data visualization and reporting 12,16 .…”

mentioning

confidence: 99%

Accuracy and efficiency of germline variant calling pipelines for human genome data

Zhao

Agafonov

Azab

et al. 2020

Sci Rep

View full text Add to dashboard Cite

Advances in next-generation sequencing technology have enabled whole genome sequencing (WGS) to be widely used for identification of causal variants in a spectrum of genetic-related disorders, and provided new insight into how genetic polymorphisms affect disease phenotypes. The development of different bioinformatics pipelines has continuously improved the variant analysis of WGS data. However, there is a necessity for a systematic performance comparison of these pipelines to provide guidance on the application of WGS-based scientific and clinical genomics. In this study, we evaluated the performance of three variant calling pipelines (GATK, DRAGEN and DeepVariant) using the Genome in a Bottle Consortium, “synthetic-diploid” and simulated WGS datasets. DRAGEN and DeepVariant show better accuracy in SNP and indel calling, with no significant differences in their F1-score. DRAGEN platform offers accuracy, flexibility and a highly-efficient execution speed, and therefore superior performance in the analysis of WGS data on a large scale. The combination of DRAGEN and DeepVariant also suggests a good balance of accuracy and efficiency as an alternative solution for germline variant detection in further applications. Our results facilitate the standardization of benchmarking analysis of bioinformatics pipelines for reliable variant detection, which is critical in genetics-based medical research and clinical applications.

show abstract

mentioning

confidence: 99%

Accuracy and efficiency of germline variant calling pipelines for human genome data

Zhao

Agafonov

Azab

et al. 2020

Sci Rep

View full text Add to dashboard Cite

show abstract

“…To accurately assess performance of bioinformatics pipelines on any dataset, the ground truth is needed (37). Simulated datasets are attractive for this reason, where features of interest (e.g.…”

Section: Discussionmentioning

confidence: 99%

Systematic benchmarking of ‘all-in-one’ microbial SNP calling pipelines

Falconer

Cuddihy

Beatson

et al. 2022

Preprint

View full text Add to dashboard Cite

Clinical and public health microbiology is increasingly utilising whole genome sequencing (WGS) technology and this has lead to the development of a myriad of analysis tools and bioinformatics pipelines. However, in order to ensure robust methodologies suitable for clinical application of this technology, accurate, reproducible, traceable and benchmarked analysis pipelines are necessary. To date, the approach to benchmarking of these has been largely ad-hoc with new pipelines benchmarked on their own datasets with limited comparisons to previously published pipelines. In this study, Snpdragon, a fast and accurate SNP calling pipeline is introduced. Written in Nextflow, Snpdragon is capable of handling small to very large and incrementally growing datasets. Snpdragon is benchmarked using previously published datasets against six other all-in-one microbial SNP calling pipelines, Lyveset, Lyveset2, Snippy, SPANDx, BactSNP and Nesoni. The effect of dataset choice on performance measures is demonstrated to highlight some of the issues associated with the current benchmarking approaches. The establishment of an agreed upon gold-standard benchmarking process for microbial variant analysis is becoming increasingly important to aid in its robust application, improve transparency of pipeline performance under different settings and direct future improvements and development of new pipelines. Snpdragon is available at https://github.com/FordeGenomics/SNPdragon.

show abstract

“…For major changes (e.g., new reportable genes, changing procedure QC metrics and acceptance criteria, addition of new concepts to reports), the AoURP will obtain FDA approval through a supplemental application to the parent IDE. 'Moderate' changes, defined as those that do not affect the validity of the data (see 21…”

Section: Change Managementmentioning

confidence: 99%

Whole genome sequencing as an investigational device for return of hereditary disease risk and pharmacogenomic results as part of the All of Us Research Program

Smith

Walker

Neben

et al. 2021

Preprint

View full text Add to dashboard Cite

The All of Us Research Program (AoURP, ‘the program’) is an initiative, sponsored by the National Institutes of Health (NIH), that aims to enroll one million people (or more) across the United States. Through repeated engagement of participants, a research resource is being created to enable a variety of future observational and interventional studies. The program has also committed to genomic data generation and returning important health-related information to participants. To do so, whole genome sequencing (WGS), variant calling processes, data interpretation, and return-of-results procedures had to be created and receive an Investigational Device Exemption (IDE) from the United States Food and Drug Administration (FDA). The performance of the entire workflow was assessed through the largest known cross-center, WGS-based, validation activity that was refined iteratively through interactions with the FDA over many months. The accuracy and precision of the WGS process as a device for the return of certain health-related genomic results was determined to be sufficient, and an IDE was granted. We present here both the process of navigating the IDE application process with the FDA and the results of the validation study as a guide to future projects which may need to follow a similar path. Future supplements to the IDE will be submitted to support additional variant classes, sample types, and any expansion to the reportable regions.

show abstract

Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays

Cited by 9 publications

References 19 publications

Accuracy and efficiency of germline variant calling pipelines for human genome data

Accuracy and efficiency of germline variant calling pipelines for human genome data

Systematic benchmarking of ‘all-in-one’ microbial SNP calling pipelines

Whole genome sequencing as an investigational device for return of hereditary disease risk and pharmacogenomic results as part of the All of Us Research Program

Contact Info

Product

Resources

About