DeepPrime2Sec: Deep Learning for Protein Secondary Structure Prediction from the Primary Sequences

Abstract

Motivation: Here we investigate deep learning-based prediction of protein secondary structure from the protein primary sequence. We study the function of different features in this task, including one-hot vectors, biophysical features, protein sequence embedding (ProtVec), deep contextualized embedding (known as ELMo), and the Position Specific Scoring Matrix (PSSM). In addition to the role of features, we evaluate various deep learning architectures including the following models/mechanisms and certain combinations: Bidirectional Long Short-Term Memory (BiLSTM), convolutional neural network (CNN), highway connections, attention mechanism, recurrent neural random fields, and gated multi-scale CNN. Our results suggest that PSSM concatenated to one-hot vectors are the most important features for the task of secondary structure prediction. Results: Utilizing the CNN-BiLSTM network, we achieved an accuracy of 69.9% and 70.4% using ensemble top-k models, for 8-class of protein secondary structure on the CB513 dataset, the most challenging dataset for protein secondary structure prediction. Through error analysis on the best performing model, we showed that the misclassification is significantly more common at positions that undergo secondary structure transitions, which is most likely due to the inaccurate assignments of the secondary structure at the boundary regions. Notably, when ignoring amino acids at secondary structure transitions in the evaluation, the accuracy increases to 90.3%. Furthermore, the best performing model mostly mistook similar structures for one another, indicating that the deep learning model inferred high-level information on the secondary structure. Availability: The developed software called DeepPrime2Sec and the used datasets are available at

WITHDRAWN: ToxVec: Deep Language Model-Based Representation Learning for Venom Peptide Classification

“…Like natural language processing scenarios, fine-tuning of language model-baed representations improved the downstream supervised task performance, which is particularly evident for small training sets. The success of automatic representation learning approaches in our experiments motivates exploring of contextualized embedding (transformers (Rao et al, 2019) or ELMo embeddings (Asgari et al, 2019b;Heinzinger et al, 2019)) as future directions.…”

EpitopeVec: Linear Epitope Prediction Using Deep Protein Sequence Embeddings

“…However, in our implementation, we proposed the scale's negative logarithm before normalization, improving its discriminative power. Sequence-based embeddings have been used successfully in protein functional/structural annotations tasks previously such as secondary structure prediction (Li and Yu, 2016;Asgari et al, 2019a), point mutations (Zhou et al, 2020), protein function prediction (Asgari and Mofrad, 2015;Zhou et al, 2019;Bonetta and Valentino, 2020), and predicting structural motifs (Liu et al, 2018). In this paper, we proposed the use of ProtVec embeddings and k-mers for linear BCE prediction improving state-of-theart performance on different datasets.…”

Evaluation of Methods for Protein Representation Learning: A Quantitative Analysis

“…Also, deep language models, such as BERT 91 and ELMO 46 were originally developed for NLP, and later employed for protein representations 23,28 . Furthermore, Convolutional Neural networks (CNNs), having the ability to learn to summarize the data with adaptive filters, have been employed to represent proteins 23,63,86,102,103 . Additionally, architectures that are capable of inferring patterns from sequential data (e.g., protein sequences) using the attention mechanism 23,55 , such as Long Short-Term Memory (LSTM) neural networks 23,28,44,104,105 and transformer based algorithms 106 , are used in representation methods.…”