A Model Study of In Silico Proficiency Testing for Clinical Next-Generation Sequencing

Duncavage, Eric J.; Abel, Haley J.; Merker, Jason D.; Bodner, John B.; Zhao, Qin; Voelkerding, Karl V.; Pfeifer, John D.

doi:10.5858/arpa.2016-0194-cp

Cited by 36 publications

(29 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These “foundational” accreditation requirements were designed to be broadly applicable to the testing of inheritable disorders, molecular oncology, and infectious diseases. Along the same lines, the feasibility of in silico proficiency testing has been demonstrated for NGS (35). Recently, high accuracy and reproducibility were shown for WGS-based microbial strain typing performed in a ring trial study involving five laboratories; the results suggested that a proficiency testing program for WGS is feasible in clinical microbiology laboratories (36).…”

Section: Introductionmentioning

confidence: 93%

Validation and Implementation of Clinical Laboratory Improvements Act-Compliant Whole-Genome Sequencing in the Public Health Microbiology Laboratory

et al. 2017

View full text Add to dashboard Cite

Public health microbiology laboratories (PHLs) are on the cusp of unprecedented improvements in pathogen identification, antibiotic resistance detection, and outbreak investigation by using whole-genome sequencing (WGS). However, considerable challenges remain due to the lack of common standards. Here, we describe the validation of WGS on the Illumina platform for routine use in PHLs according to Clinical Laboratory Improvements Act (CLIA) guidelines for laboratory-developed tests (LDTs). We developed a validation panel comprising 10 Enterobacteriaceae isolates, 5 Gram-positive cocci, 5 Gram-negative nonfermenting species, 9 Mycobacterium tuberculosis isolates, and 5 miscellaneous bacteria. The genome coverage range was 15.71× to 216.4× (average, 79.72×; median, 71.55×); the limit of detection (LOD) for single nucleotide polymorphisms (SNPs) was 60×. The accuracy, reproducibility, and repeatability of base calling were >99.9%. The accuracy of phylogenetic analysis was 100%. The specificity and sensitivity inferred from multilocus sequence typing (MLST) and genome-wide SNP-based phylogenetic assays were 100%. The following objectives were accomplished: (i) the establishment of the performance specifications for WGS applications in PHLs according to CLIA guidelines, (ii) the development of quality assurance and quality control measures, (iii) the development of a reporting format for end users with or without WGS expertise, (iv) the availability of a validation set of microorganisms, and (v) the creation of a modular template for the validation of WGS processes in PHLs. The validation panel, sequencing analytics, and raw sequences could facilitate multilaboratory comparisons of WGS data. Additionally, the WGS performance specifications and modular template are adaptable for the validation of other platforms and reagent kits.

show abstract

Section: Introductionmentioning

confidence: 93%

Validation and Implementation of Clinical Laboratory Improvements Act-Compliant Whole-Genome Sequencing in the Public Health Microbiology Laboratory

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Nevertheless, the traditional, individual target-specific approach to validation is not feasible for metagenomic sequencing tests because it is impossible to include all potential organism types and variants in validation studies. Similar challenges in molecular genetic pathology have led the College of American Pathologists to recommend methods-based or in silico proficiency testing 29,30 as an alternative. This approach is based on a combination of real sequencing data with in silico-generated or modified sequences for proficiency testing, which allows for more in-depth testing of algorithms and sequence databases, because a large diversity of specimens can be generated in silico and critical data analysis steps can be performed with fewer resources.…”

Section: Assay Validationmentioning

confidence: 99%

Validation of Metagenomic Next-Generation Sequencing Tests for Universal Pathogen Detection

Salais

Chiu

Miller

et al. 2017

Archives of Pathology &Amp; Laboratory Medicine

419

373

View full text Add to dashboard Cite

- Although laboratory and data analysis workflows are still complex, metagenomic NGS tests for infectious diseases are increasingly being validated in clinical laboratories. Many parallels exist to NGS tests in other fields. Nevertheless, specimen preparation, rapidly evolving data analysis algorithms, and incomplete reference sequence databases are idiosyncratic to the field of microbiology and often overlooked.

show abstract

“…The study showed a high degree of concordance for identification of single-nucleotide variants at varying allele fractions, whereas a lower degree of concordance was demonstrated for the identification of insertions and deletions. Second, Duncavage et al 42 reported on a pilot study using in silico manipulated FASTQ files generated from two commercially available reagent sets for NGS of solid tumors. In this study, a variety of sequence variants were introduced into FASTQ files at varying allele frequencies and distributed to the pilot laboratories, with results demonstrating that laboratories were able to process the files and identify variants, establishing the feasibility of the approach.…”

Section: Recommendation 14: In Silico Validation Can Be Used To Supplmentioning

confidence: 99%

Standards and Guidelines for Validating Next-Generation Sequencing Bioinformatics Pipelines

Roy

Coldren²,

Karunamurthy

et al. 2018

The Journal of Molecular Diagnostics

Self Cite

357

129

View full text Add to dashboard Cite

Bioinformatics pipelines are an integral component of next-generation sequencing (NGS). Processing raw sequence data to detect genomic alterations has significant impact on disease management and patient care. Because of the lack of published guidance, there is currently a high degree of variability in how members of the global molecular genetics and pathology community establish and validate bioinformatics pipelines. Improperly developed, validated, and/or monitored pipelines may generate inaccurate results that may have negative consequences for patient care. To address this unmet need, the Association of Molecular Pathology, with organizational representation from the College of American Pathologists and the American Medical Informatics Association, has developed a set of 17 best practice consensus recommendations for the validation of clinical NGS bioinformatics pipelines. Recommendations include practical guidance for laboratories regarding NGS bioinformatics pipeline design, development, and operation, with additional emphasis on the role of a properly trained and qualified molecular professional to achieve optimal NGS testing quality.

show abstract

A Model Study of In Silico Proficiency Testing for Clinical Next-Generation Sequencing

Abstract: -The results indicate that in silico proficiency testing is a feasible approach for methods-based proficiency testing, and demonstrate that the sensitivity and specificity of current next-generation sequencing bioinformatics across clinical laboratories are high.

Cited by 36 publications

References 20 publications

Validation and Implementation of Clinical Laboratory Improvements Act-Compliant Whole-Genome Sequencing in the Public Health Microbiology Laboratory

Validation and Implementation of Clinical Laboratory Improvements Act-Compliant Whole-Genome Sequencing in the Public Health Microbiology Laboratory

Validation of Metagenomic Next-Generation Sequencing Tests for Universal Pathogen Detection

Standards and Guidelines for Validating Next-Generation Sequencing Bioinformatics Pipelines

Contact Info

Product

Resources

About