Evaluating the informativeness of deep learning annotations for human complex diseases

Abstract

Deep learning models have shown great promise in predicting genome-wide regulatory effects from DNA sequence, but their informativeness for human complex diseases and traits is not fully understood. Here, we evaluate the disease informativeness of two types of deep learning annotations: (1) variant-level annotations (based on the reference allele), assessing whether they are more informative for complex disease than the underlying experimental data used to train the predictive models; and (2) allelic-effect annotations (absolute value of the predicted difference between reference and variant alleles), which have been a major focus of recent work. In each case, we primarily consider annotations constructed using two previously trained deep learning models, DeepSEA and Basenji. We apply stratified LD score regression (S-LDSC) to 41 independent diseases and complex traits (average N =320K) to evaluate each annotation's informativeness for disease heritability conditional on a broad set of coding, conserved, regulatory and LDrelated annotations from the baseline-LD model and other sources; as a secondary metric, we also evaluate the accuracy of models that incorporate deep learning annotations in predicting disease-associated or fine-mapped SNPs. We aggregated annotations across all tissues (resp. blood cell types or brain tissues) in metaanalyses across all 41 traits (resp. 11 blood-related traits or 8 brain-related traits). Variant-level annotations, despite being highly enriched for disease heritability, produced no conditionally significant results in meta-analyses across all 41 traits or 11 blood-related traits, but brain-specific DeepSEA-H3K4me3 and Basenji-H3K27ac annotations were conditionally significant in meta-analyses across 8 brain-related traits; a sequence motif analysis suggests that these annotations could be capturing unique information about nucleosome occupancy. Allelic-effect annotations were also highly enriched for disease heritability, and produced conditionally significant results for Basenji-H3K4me3 in meta-analyses across all 41 traits and brain-specific Basenji-H3K4me3 in meta-analyses across 8 brain-related traits. We conclude that deep learning models are informative for disease, but their informativeness cannot be inferred from metrics based on their accuracy in predicting regulatory annotations.

SNP-to-gene linking strategies reveal contributions of enhancer-related and candidate master-regulator genes to autoimmune disease

“…We conditioned all analyses on a "baseline-LD-deep model" defined by 86 coding, conserved, regulatory and LD-related annotations from the baseline-LD model (v2.1) 34,35 and 14 additional jointly significant annotations from ref. 40 : 1 non-tissue-specific allelic-effect Basenji annotation, 3 Roadmap and 5 ChromHMM annotations and 5 other annotations (100 annotations total) ( Table ?? and Table ??…”

Integrative approaches to improve the informativeness of deep learning models for human complex diseases

“…Basenji employs a Poisson likelihood model to analyze 130kb of human reference sequence around each SNP using dilated convolutional layers. Our previous work 16 focused on unsigned (absolute) allelic-effect annotations for DNase and three histone marks, H3K27ac, H3K4me1 and H3K4me3 (associated with active enhancers and promoters). Here, we integrate signed allelic-effect annotations with other types of data -fine-mapped SNPs, SNPs linked to genes, and gene expression -to generate more disease-informative unsigned annotations.…”

“…The boosted allelic-effect annotations derived from DeepBoost (DeepSEA∆-boosted and Basenji∆-boosted; we use ∆ to denote allelic-effect annotations) were only moderately correlated with allelic-effect annotations derived from a simple maximum of published deep learning annotations across 4 chromatin marks (DNase, H3K27ac, H3K4me1 and H3K4me3) and 27 blood cell types, as in ref. 16 (DeepSEA∆-published and Basenji∆-published) (average r = 0.35) ( Figure S2). We broadly investigated which features of the DeepSEA and Basenji models contributed the most to the DeepSEA∆-boosted and Basenji∆-boosted annotations by applying Shapley Additive Explanation (SHAP) 36 , a widely used tool for interpreting machine-learning models.…”

“…For each SNP with minor allele count ≥ 5 in 1000 Genomes, we applied the pre-trained DeepSEA and Basenji models to the surrounding DNA sequence to compute both the prediction (at reference allele) and the predicted difference in probability between the reference and the alternate alleles. We call these the variant-level annotations and allelic-effect annotations respectively; this naming convention has been used previously 16 . The allelic-effect annotations are more interesting from a biological perspective as they are specific to a sequence-based predictive model like these deep learning models.…”

“…Additional annotations of baseline-LD-deep model:List of 14 jointly significant annotations from ref. 16 added to the baseline-LD model to create the baseline-LD-deep model. They include 1 non-tissue-specific Basenji allelic-effect annotation, 3 Roadmap annotations, 5 ChromHMM annotations and 5 other annotations.…”