SAINT: self-attention augmented inception-inside-inception network improves protein secondary structure prediction

Abstract

Motivation Protein structures provide basic insight into how they can interact with other proteins, their functions and biological roles in an organism. Experimental methods (e.g., X-ray crystallography, nuclear magnetic resonance spectroscopy) for predicting the secondary structure (SS) of proteins are very expensive and time consuming. Therefore, developing efficient computational approaches for predicting the secondary structure of protein is of utmost importance. Advances in developing highly accurate SS prediction methods have mostly been focused on 3-class (Q3) structure prediction. However, 8-class (Q8) resolution of secondary structure contains more useful information and is much more challenging than the Q3 prediction. Results We present SAINT, a highly accurate method for Q8 structure prediction, which incorporates self-attention mechanism (a concept from natural language processing) with the Deep Inception-Inside-Inception (Deep3I) network in order to effectively capture both the short-range and long-range interactions among the amino acid residues. SAINT offers a more interpretable framework than the typical black-box deep neural network methods. Through an extensive evaluation study, we report the performance of SAINT in comparison with the existing best methods on a collection of benchmark datasets, namely, TEST2016, TEST2018, CASP12 and CASP13. Our results suggest that self-attention mechanism improves the prediction accuracy and outperforms the existing best alternate methods. SAINT is the first of its kind and offers the best known Q8 accuracy. Thus, we believe SAINT represents a major step towards the accurate and reliable prediction of secondary structures of proteins. Availability SAINT is freely available as an open source project at https://github.com/SAINTProtein/SAINT. Supplementary information Supplementary data are available at Bioinformatics online.

EGRET: Edge Aggregated Graph Attention Networks and Transfer Learning Improve Protein-Protein Interaction Site Prediction

“…We investigated the interpretability of EGRET to provide insights on how the architecture is making decisions. Some recent studies demonstrated partial interpretability of deep neural networks for solving various problems in computational biology (in particular, see Uddin et al (2020) and Vig et al (2020)).…”

OPUS-X: An Open-Source Toolkit for Protein Torsion Angles, Secondary Structure, Solvent Accessibility, Contact Map Predictions, and 3D Folding

“…In this research, we use 5 independent test sets to evaluate the performance of different approaches. CASP-FM (56), collected by SAINT 29 , contains 10 template-free modeling (FM) targets from CASP13, 22 FM targets from CASP12, 16 FM targets from CASP11, and 8 FM targets from CASP10. CASP13 (26) contains 26 FM targets from CASP13.…”

Protein Secondary Structure Prediction With a Reductive Deep Learning Method

“…For instance, π-helix is found abundant and associated with activities in some special proteins (Cooley et al, 2010 ). As a result, over the few years many efforts have been made, trying to solve the Q8 prediction problem, which is much more complicated and challenging (Li and Yu, 2016 ; Wang et al, 2016 ; Fang et al, 2017 ; Heffernan et al, 2017 ; Zhang et al, 2018 ; Krieger and Kececioglu, 2020 ; Uddin et al, 2020 ; Guo et al, 2021 ) If not otherwise specified, the models discussed in this paper are non-template based. The Q8 prediction accuracy has reached 70% and at present the best record is 77.73% (Uddin et al, 2020 ).…”

“…Over the past few decades, a variety of state-of-the-art methods have been developed to improve Q3 or Q8 prediction accuracy and most progresses are contributed by machine learning based models (Li and Yu, 2016 ; Wang et al, 2016 ; Fang et al, 2017 ; Heffernan et al, 2017 ; Zhang et al, 2018 ; Krieger and Kececioglu, 2020 ; Uddin et al, 2020 ; Guo et al, 2021 ) So far as we know, the predictive power of a machine learning model is governed mainly by two elements, namely feature representation and algorithm. For instance, the introduction of sequence evolutionary profiles from multiple-sequence alignment (Rost and Sander, 1993 ), such as position-specific scoring matrices (PSSM) (Jones, 1999 ), improves prediction accuracy significantly (Zhou and Troyanskaya, 2014 ).…”

“…CNN and RNN architectures were integrated for the first time in the DCRNN model to predict protein secondary structures (Li and Yu, 2016 ; Zhang et al, 2018 ). Some models employ different deep learning architectures, such as the deep conditioned neural field (DeepCNF) (Wang et al, 2016 ) and the deep inception-inside-inception network (Deep3I) (Fang et al, 2017 ; Uddin et al, 2020 ). In particular, the model SAINT that incorporates self-attention mechanism and Deep3I provides up-to-date the best Q8 prediction accuracy (Uddin et al, 2020 ).…”