A-Prot: protein structure modeling using MSA transformer

Abstract

Background The accuracy of protein 3D structure prediction has been dramatically improved with the help of advances in deep learning. In the recent CASP14, Deepmind demonstrated that their new version of AlphaFold (AF) produces highly accurate 3D models almost close to experimental structures. The success of AF shows that the multiple sequence alignment of a sequence contains rich evolutionary information, leading to accurate 3D models. Despite the success of AF, only the prediction code is open, and training a similar model requires a vast amount of computational resources. Thus, developing a lighter prediction model is still necessary. Results In this study, we propose a new protein 3D structure modeling method, A-Prot, using MSA Transformer, one of the state-of-the-art protein language models. An MSA feature tensor and row attention maps are extracted and converted into 2D residue-residue distance and dihedral angle predictions for a given MSA. We demonstrated that A-Prot predicts long-range contacts better than the existing methods. Additionally, we modeled the 3D structures of the free modeling and hard template-based modeling targets of CASP14. The assessment shows that the A-Prot models are more accurate than most top server groups of CASP14. Conclusion These results imply that A-Prot accurately captures the evolutionary and structural information of proteins with relatively low computational cost. Thus, A-Prot can provide a clue for the development of other protein property prediction methods.

Fast, accurate antibody structure prediction from deep learning on massive set of natural antibodies

“…Concurrent with advances in structure prediction, self-supervised learning on massive sets of unlabeled protein sequences has shown remarkable utility across protein modeling tasks (17, 18). Embeddings from transformer encoder models trained for masked language modeling have been used for variant prediction (19), evolutionary analysis (20, 21), and as features for protein structure prediction (22, 23). Auto-regressive transformer models have been used to generate functional proteins entirely from sequence learning (24).…”

Fast, accurate antibody structure prediction from deep learning on massive set of natural antibodies

“…Concurrent with advances in structure prediction, self-supervised learning on massive sets of unlabeled protein sequences has shown remarkable utility across protein modeling tasks 18,19 . Embeddings from transformer encoder models trained for masked language modeling have been used for variant prediction 20 , evolutionary analysis 21,22 , and as features for protein structure prediction 23,24 . Auto-regressive transformer models have been used to generate functional proteins entirely from sequence learning 25 .…”

Foundation models in molecular biology

“…A-Port (Hong et al . 2022 ) performs residue contact prediction using MSA-Transformer and inputs the predicted pairs of contacting residues into PyRosetta for protein structure prediction. The analysis results show that the quality of structures predicted by A-Port exceeds the current best structure prediction methods, but it is still far from solving the problem of protein structure prediction.…”