An Extensive Sequence Dataset of Gold-Standard Samples for Benchmarking and Development

Baid, Gunjan; Nattestad, Maria; Kolesnikov, Alexey; Goel, Sidharth; Yang, Howard H.; Chang, Pi-Chuan; Carroll, Andrew

doi:10.1101/2020.12.11.422022

Cited by 21 publications

(19 citation statements)

References 39 publications

(45 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Exome models are trained on Agilent SureSelect v7, IDT-xGen, and Truseq capture kits. This data has been previously described and released 33 . DeepTrio is trained on examples from chromosomes 1-19.…”

Section: Modifying Deepvariant To Call Triosmentioning

confidence: 99%

See 1 more Smart Citation

DeepTrio: Variant Calling in Families Using Deep Learning

Kolesnikov

Goel

Nattestad

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Every human inherits one copy of the genome from their mother and another from their father. Parental inheritance helps us understand the transmission of traits and genetic diseases, which often involve de novo variants and rare recessive alleles. Here we present DeepTrio, which learns to analyze child-mother-father trios from the joint sequence information, without explicit encoding of inheritance priors. DeepTrio learns how to weigh sequencing error, mapping error, and de novo rates and genome context directly from the sequence data. DeepTrio has higher accuracy on both Illumina and PacBio HiFi data when compared to DeepVariant. Improvements are especially pronounced at lower coverages (with 20x DeepTrio roughly equivalent to 30x DeepVariant). As DeepTrio learns directly from data, we also demonstrate extensions to exome calling solely by changing the training data. DeepTrio includes pre-trained models for Illumina WGS, Illumina exome, and PacBio HiFi.

show abstract

Section: Modifying Deepvariant To Call Triosmentioning

confidence: 99%

“…The generation of sequencing data for training is described in detail in Baid et al 2020 33 and the WGS and PacBio evaluation data in Olson et al 2020 22 . In summary, all WGS and exome runs were conducted with 151-bp paired-end reads at 50x intended coverage from NovaSeq and HiSeqX platforms.…”

Section: Generation Of Sequencing Datamentioning

confidence: 99%

DeepTrio: Variant Calling in Families Using Deep Learning

Kolesnikov

Goel

Nattestad

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…We further evaluated the performance of the models using two whole-exome sequencing (WES) datasets from a recently released set of genome and exome data for HG003 [28] (Figure 2c, Tables S2 and S3). Both datasets were aligned to GRCh37 and evaluated using the GIAB v4.2.1 truth set.…”

Section: Population Information Improves Variant Calling Performancementioning

confidence: 99%

“…We trained the model following the procedure described in [2], with additional Illumina WGS datasets included [28]. Variants in chromosomes 1 to 19 are used as the training examples, and those in chromosome 21 and 22 are used for tuning.…”

Section: Model Trainingmentioning

confidence: 99%

Improving variant calling using population data and deep learning

Chen

Kolesnikov

Goel

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Large-scale population variant data is often used to filter and aid interpretation of variant calls in a single sample. These approaches do not incorporate population information directly into the process of variant calling, and are often limited to filtering which trades recall for precision. In this study, we modify DeepVariant to add a new channel encoding population allele frequencies from the 1000 Genomes Project. We show that this model reduces variant calling errors, improving both precision and recall. We assess the impact of using population-specific or diverse reference panels. We achieve the greatest accuracy with diverse panels, suggesting that large, diverse panels are preferable to individual populations, even when the population matches sample ancestry. Finally, we show that this benefit generalizes to samples with different ancestry from the training data even when the ancestry is also excluded from the reference panel.

show abstract

“…This would improve the models and increase robustness of the best-performing variant callers on different ethnical backgrounds, and increase the opportunity for cross-benchmarking. Recently, a new set of highquality reference datasets have been generated for benchmarking of variant callers (Baid et al, 2020), and evaluation of several variant callers' performance on these data corroborates the results was not certified by peer review) is the author/funder. All rights reserved.…”

Section: Discussionmentioning

confidence: 80%

Systematic benchmark of state-of-the-art variant calling pipelines identifies major factors affecting accuracy of coding sequence variant discovery

Abasov¹,

et al. 2021

Preprint

View full text Add to dashboard Cite

Accurate variant detection in the coding regions of the human genome is a key requirement for molecular diagnostics of Mendelian disorders. Efficiency of variant discovery from next-generation sequencing (NGS) data depends on multiple factors, including reproducible coverage biases of NGS methods and the performance of read alignment and variant calling software. In this work, we systematically evaluated the performance of 4 popular short read aligners (Bowtie2, BWA, Isaac, and Novoalign) and 6 variant calling and filtering methods (based on DeepVariant, Genome Analysis ToolKit (GATK), FreeBayes, and Strelka2) using a set of 10 gold standard WES and WGS datasets. Our results suggest that Bowtie2 performs significantly worse than other aligners and should not be used for medical variant calling. When other aligners were used, the accuracy of variant discovery mostly depended on the variant caller and not the read aligner. Among the tested variant callers, DeepVariant consistently showed the best performance and the highest robustness, with other state-of-the-art tools, i.e. Strelka2 and GATK with 1D convolutional neural network variant scoring, also showing high performance on both WES and WGS data. The results show surprisingly large differences in the performance of cutting-edge tools even on high confidence regions of the coding genome. This highlights the importance of regular benchmarking of quickly evolving tools and pipelines. Finally, we discuss the need for a more diverse set of gold standard genomes (e.g. of African, Hispanic, or mixed ancestry) that would allow to control for deep learning model overfitting. For similar reasons there is a need for better variant caller assessment in the repetitive regions of the coding genome.

show abstract

An Extensive Sequence Dataset of Gold-Standard Samples for Benchmarking and Development

Cited by 21 publications

References 39 publications

DeepTrio: Variant Calling in Families Using Deep Learning

DeepTrio: Variant Calling in Families Using Deep Learning

Improving variant calling using population data and deep learning

Systematic benchmark of state-of-the-art variant calling pipelines identifies major factors affecting accuracy of coding sequence variant discovery

Contact Info

Product

Resources

About