Global Importance Analysis: An Interpretability Method to Quantify Importance of Genomic Features in Deep Neural Networks

Abstract

Deep neural networks have demonstrated improved performance at predicting the sequence specificities of DNA- and RNA-binding proteins compared to previous methods that rely on k-mers and position weight matrices. For model interpretability, attribution methods have been employed to reveal learned patterns that resemble sequence motifs. First-order attribution methods only quantify the independent importance of single nucleotide variants in a given sequence – it does not provide the effect size of motifs (or their interactions with other patterns) on model predictions. Here we introduce global importance analysis (GIA), a new model interpretability method that quantifies the population-level effect size that putative patterns have on model predictions. GIA provides an avenue to quantitatively test hypotheses of putative patterns and their interactions with other patterns, as well as map out specific functions the network has learned. As a case study, we demonstrate the utility of GIA on the computational task of predicting RNA-protein interactions from sequence. We first introduce a new convolutional network, we call ResidualBind, and benchmark its performance against previous methods on RNAcompete data. Using GIA, we then demonstrate that in addition to sequence motifs, ResidualBind learns a model that considers the number of motifs, their spacing, and sequence context, such as RNA secondary structure and GC-bias.

Discovering differential genome sequence activity with interpretable and efficient deep learning

“…This in combination with the large number overlap of Nrf1 and Tcf12 binding sites suggests Nrf1 and Tcf12 binding may be regulated by logic beyond the presence or absence of their DNA binding motifs. We also show that under class imbalanced data, EPE and DEPE generate robust estimates of transcription factor effects in contrast to Global Importance Analysis (15), which takes a related approach but uses a difference instead of a ratio to estimate the effect of a pattern (Fig S2).…”

“…In these cases, explicitly training models on each differential comparison between cell types would be computational expensive and time intensive. We also note that Expected Pattern Effect resembles other methods for extracting pattern effects from deep neural networks (3,15), except that we compute Expected Pattern Effect to permit the comparison of pattern effects between conditions, which is important for identification of cell type-specific or condition-specific sequence features and show that our use of ratio to compare effects is more robust to analysis of cell type-specific transcription factor activity under class imbalance.…”

“…We also avoid using package managers such as pip and conda which are present with frameworks such as Selene 20 , Kipoi 21 , and pyyster 22 , as problems with incompatibility can still arise with typical package managers as a consequence of python, package manager, and operating system updates. We note that Expected Pattern Effect resembles other methods for extracting pattern effects from deep neural networks 3,13 , except that we compute Expected Pattern Effect such that we can compare pattern effects between conditions, which is important for identification of cell typespecific or condition-specific sequence features. While DeepAccess was developed specifically for identifying cell type-specific sequence features from chromatin accessibility, Differential Expected Pattern Effect can be used to discover condition-specific sequence features from many types of experimental genome-wide sequencing data.…”

“…We note that Expected Pattern Effect resembles other methods for extracting pattern effects from deep neural networks 3,13 , except that we compute Expected Pattern Effect such that we can compare pattern effects between conditions, which is important for identification of cell type-specific or condition-specific sequence features.…”

Thermodynamic modeling reveals widespread multivalent binding by RNA-binding proteins

“…More recently, a new wave of algorithms have been introduced that use deep neural networks to predict RBP binding sites ( Alipanahi et al , 2015 ; Ghanbari and Ohler, 2020 ; Grønning et al , 2020 ; Pan and Shen, 2018 ; Yan and Zhu, 2020 ). One challenge is to explain what these complex models have learned, although recently a multitude of methods for interpreting the learned models have been developed, for instance, based on in silico mutagenesis, predictions on synthetic sequences, gradient tracing and analyzing the convolutional filters ( Alipanahi et al , 2015 ; Ghanbari and Ohler, 2020 ; Koo et al , 2020 ; Pan and Shen, 2018 ). However, with the increasing number of model parameters and network complexity, the risk grows that such models could also learn experimental biases in the datasets.…”