On the Depth of Deep Learning Models for Splice Site Identification

Abstract

The success of deep learning has been shown in various fields including computer vision, speech recognition, natural language processing and bioinformatics. The advance of Deep Learning in Computer Vision has been an important source of inspiration for other research fields. The objective of this work is to adapt known deep learning models borrowed from computer vision such as VGGNet, Resnet and AlexNet for the classification of biological sequences. In particular, we are interested by the task of splice site identification based on raw DNA sequences. We focus on the role of model architecture depth on model training and classification performance.We show that deep learning models outperform traditional classification methods (SVM, Random Forests, and Logistic Regression) for large training sets of raw DNA sequences. Three model families are analyzed in this work namely VGGNet, AlexNet and ResNet. Three depth levels are defined for each model family. The models are benchmarked using the following metrics: Area Under ROC curve (AUC), Number of model parameters, number of floating operations. Our extensive experimental evaluation show that shallow architectures have an overall better performance than deep models. We introduced a shallow version of ResNet, named S-ResNet. We show that it gives a good trade-off between model complexity and classification performance. Author summaryDeep Learning has been widely applied to various fields in research and industry. It has 1 been also succesfully applied to genomics and in particular to splice site identification. 2We are interested in the use of advanced neural networks borrowed from computer

Human exons and introns classification using pre-trained Resnet-50 and GoogleNet models and 13-layers CNN model

“…Our obtained results are comparable with results in 28 where, based on a simplified version of ResNet, they suggested a novel model (S-ResNet). S-ResNet differs from Resnet in that it places a shortcut link at each layer of the convolution process rather than after a block made up of two convolution layers.…”

“… Spec. Resnet-50 92% 94.2% 90.4% GoogleNet 91.5% 95.9% 90.2% 13-layers CNN 91.6% 92.4% 94.2% S-Resnet 28 93.5% 92.9% 93.8% ANFIS 29 88.88% 75% 100% …”

DASSI: Differential Architecture Search for Splice Identification from DNA Sequences

“…The challenges involved in performing manual feature extraction and model training led to development of models using Artificial Neural Network (ANN) [31,32] that performed automated feature representation. Many DL architectures were used and developed for splice site prediction based on CNN [33,34,35,36,37], RNN [13,38], Restricted Boltzmann Machines (RBM) [39], Autoencoders [40,41] and Deep Belief Networks [39]. Although these DL architectures have removed the burden of manual feature extraction, they are still time consuming to train and a much deeper knowledge on SS associated functions and evolution has been strongly urged.…”

Differential Architecture Search in Deep Learning for DNA Splice Site Classification

“…The challenges involved in performing manual feature extraction and model training has led to a demand for DL based computational methods that performed automated feature representation. Many DL architectures were used and developed for the splice site prediction based on CNN [25,26], RNN [13,27], Restricted Boltzmann Machines (RBM) [28], Autoencoders [29,30] and Deep Belief Networks [28]. Although these DL architectures have removed the burden of manual feature extraction, they are still time consuming to train and a much deeper knowledge on splice sites-associated functions and evolution has been strongly urged.…”