Effect of sequence padding on the performance of deep learning models in archaeal protein functional prediction

Abstract

The use of raw amino acid sequences as input for deep learning models for protein functional prediction has gained popularity in recent years. This scheme obliges to manage proteins with different lengths, while deep learning models require same-shape input. To accomplish this, zeros are usually added to each sequence up to a established common length in a process called zero-padding. However, the effect of different padding strategies on model performance and data structure is yet unknown. We propose and implement four novel types of padding the amino acid sequences. Then, we analysed the impact of different ways of padding the amino acid sequences in a hierarchical Enzyme Commission number prediction problem. Results show that padding has an effect on model performance even when there are convolutional layers implied. Contrastingly to most of deep learning works which focus mainly on architectures, this study highlights the relevance of the deemed-of-lowimportance process of padding and raises awareness of the need to refine it for better performance. The code of this analysis is publicly available at https ://githu b.com/b2sla b/paddi ng_bench mark. Since the breakthrough of deep learning (DL) 1 , deep neural networks are being successfully applied in computational biology 2,3. This is due to their capacity for automatically extracting meaningful features from raw data 4. Specifically, DL is useful in the context of biological sequences, such as proteins or RNA, because it can learn directly from the sequence and hence, capture nonlinear dependencies and interaction effects. Some examples of applications of DL on biological sequences include prediction of specifities of DNA and RNA binding proteins 5 , DNA function quantification 6 , de novo peptide design 7 , detection of conserved DNA fragments 8 , prediction of protein associated GO terms 9 or quantification of the impact of genetic variation on gene regulatory mechanisms 3. The specific DL architectures able to leverage the inner structure of sequential biological data are Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN). CNNs entail translational invariance 10 and can be used to find relevant patterns with biological meaning 5,8,11,12. For their part, bidirectional RNNs (and the derived Long Short-Term Memory and Gated Recurrent Units) are appropiate for modelling biological sequences since they are suited for data with a sequential but non-causal structure, variable length, and long-range dependencies 13-16. Both architectures are usually combined, as in DEEPre 17 , where a CNN-RNN model performs a hierarchical classification of enzymes. Proteins are long linear sequences constituted by amino acid residues attached covalently. These amino acid residues are represented by letters that cannot be directly processed by the mathematical operations used by DL models. Choosing how to digitally encode amino acids is a crucial step in this context, since it can affect to the overall performance of the models 18. A comprehensive review and a...

xDeep-AcPEP: Deep Learning Method for Anticancer Peptide Activity Prediction Based on Convolutional Neural Network and Multitask Learning

“…Therefore, we selected CT-padding as the default padding strategy in the experiments that followed and in the training of the final production models. Though it is unclear the reason why CT-padding performs better, our finding is in agreement with the study of Lopez-del Rio et al who compared eight different padding strategies for protein functional prediction using one-hot encoding as the feature. They confirmed that postpadding (i.e., CT-padding here) outperforms the other padding types, including prepadding (i.e., NT-padding here), middle-padding, stratified-padding, extreme-padding, random-padding, zoom-padding, and augmented-padding for convolutional architectures.…”

Pop-In Identification in Nanoindentation Curves with Deep Learning Algorithms

“…We want to highlight that the overall accuracy of the model (~93%) could have been improve by employing a larger dataset and using more materials. In addition, methodologies related to the treatment of the data, such as noise analysis (reduction or inclusion) [38,39], resampling and smoothing [40], and padding configurations [41] could have enhanced the model's accuracy.…”

Impact of Data Preparation in Freezing of Gait Detection Using Feature-Less Recurrent Neural Network

“…The analogy between these fields motivates the applications of the Natural Language Processing (NLP) techniques to FOG detection by changing the length of the sequence with minimum impact on the neural network. When applying machine learning to NLP and protein functional prediction the issue of variable sequence length is solved by adding pad values to the sequences to ensure sequences with the same length [26][27] [28][29][30] [31]. Zero padding is frequently used in the NLP and protein prediction and the padding operation concatenates a vector of zero values to the measured sequences [30][31] [32][33] [34].…”