Evaluating deep learning for predicting epigenomic profiles

Abstract

Deep learning has been successful at predicting epigenomic profiles from DNA sequences. Most approaches frame this task as a binary classification relying on peak callers to define functional activity. Recently, quantitative models have emerged to directly predict the experimental coverage values as a regression. As new models continue to emerge with different architectures and training configurations, a major bottleneck is forming due to the lack of ability to fairly assess the novelty of proposed models and their utility for downstream biological discovery. Here we introduce a unified evaluation framework and use it to compare various binary and quantitative models trained to predict chromatin accessibility data. We highlight various modeling choices that affect generalization performance, including a downstream application of predicting variant effects. In addition, we introduce a robustness metric that can be used to enhance model selection and improve variant effect predictions. Our empirical study largely supports that quantitative modeling of epigenomic profiles leads to better generalizability and interpretability.

Correcting gradient-based interpretations of deep neural networks for genomics

“…9). We also found that gradient correction worked well for various CNNs trained to predict quantitative levels of normalized read-coverage of 15 ATAC-seq datasets at base-resolution 19 (Fig. 2e, Supplementary Fig.…”

“…We acquired the test data and the trained CNN-base and CNN-32 models with exponential activations and ReLU activations from Ref. 19 ; a total of 4 models. Each CNN takes as input 2kb length sequences and outputs a prediction of normalized read-coverage for 15 ATAC-seq bigWig tracks (i.e.…”

Towards In-Silico CLIP-seq: Predicting Protein-RNA Interaction via Sequence-to-Signal Learning

“…In the context of BPNet [2], the authors therefore performed peak calling on ChIP-nexus data to select a set of regions highly enriched in count signal. However, recent work by Toneyan et al [51] suggests that peak callers select sites too conservatively, which may result in under-fitting of sequence-to-signal models.…”

EvoAug: improving generalization and interpretability of genomic deep neural networks with evolution-inspired data augmentations

“…With image data, basic affine transformations can translate, magnify, or rotate an image without changing its label. For genomics, the available neutral augmentations are reverse-complement transformations 16 and small random translations of the input sequence 17, 18 . With the finite size of experimental data and a paucity of augmentation methods, there exist only limited strategies to promote generalization for genomic DNNs.…”

“…An important downstream application of genomic DNNs is scoring the functional consequences of mutations. Following previous procedures 2, 17 , we compared model predictions with saturation mutagenesis of 15 cis -regulatory elements measured experimentally with a massively parallel reporter assay – data collected through the CAGI5 Challenge 21 . As expected, EvoAugtrained DNNs outperformed standard training on this out-of-distribution generalization task (Fig.…”