Perturbation robustness analyses reveal important parameters in variant interpretation pipelines

Abstract

<jats:title>Abstract</jats:title><jats:sec><jats:title>Motivation</jats:title><jats:p>Genome sequencing is being used routinely in clinical and research applications, but subsequent variant interpretation pipelines can vary widely. A systematic approach for exploring parameter choices and selection plays an important role in designing robust pipelines for specific clinical applications.</jats:p></jats:sec><jats:sec><jats:title>Results</jats:title><jats:p>We present a framework to be applied in scenarios with limited data whereby expert knowledge informs pipeline refinement. Starting from initial reference variant interpretation pipelines with commonly used parameters, we derived pipelines by perturbing the parameters one by one to determine which parameters can yield meaningful changes in a pipeline’s performance. We updated the reference pipeline by fixing the value of parameters which have small impact on the pipeline’s performance. Then we conducted new rounds of perturbation as the process converged, yielding a stable pipeline which is robust. We applied the framework for genetic disease prediction in de-identified exomes from a cohort of 138 individuals with rare Mendelian inborn errors of metabolism (IEMs) and systematically explored how perturbing different parameters affected the pipeline’s sensitivity and specificity. For this application, we perturbed commonly used parameters in variant interpretation pipelines, including choices of genes, variant callers, transcript models, databases of allele frequencies, databases of curated disease variants, and tools for variant impact prediction. Our analyses showed that choice of variant callers, variant impact prediction tools, MAF threshold, and MAF databases can meaningfully alter results from a pipeline. This work informs the development of exome analysis pipelines designed for newborn metabolic disorder screening and suggests the general application of perturbation analysis in genome interpretation pipeline design.</jats:p></jats:sec>

The Use of Whole Genome and Exome Sequencing for Newborn Screening: Challenges and Opportunities for Population Health

“…Because most of the population will be unaffected in rare diseases typically screened for in the newborn period, even a 1% false positive rate for a screening test can translate to a large burden of false positive cases that require follow-up. Therefore, systematic exploration of parameters in analysis pipelines is necessary to achieve a balance between sensitivity and specificity that is best suited for screening ( 103 ).…”