Unsupervised protein embeddings outperform hand-crafted sequence and structure features at predicting molecular function

Abstract

Motivation Protein function prediction is a difficult bioinformatics problem. Many recent methods use deep neural networks to learn complex sequence representations and predict function from these. Deep supervised models require a lot of labeled training data which are not available for this task. However, a very large amount of protein sequences without functional labels is available. Results We applied an existing deep sequence model that had been pre-trained in an unsupervised setting on the supervised task of protein molecular function prediction. We found that this complex feature representation is effective for this task, outperforming hand-crafted features such as one-hot encoding of amino acids, k-mer counts, secondary structure and backbone angles. Also, it partly negates the need for complex prediction models, as a two-layer perceptron was enough to achieve competitive performance in the third Critical Assessment of Functional Annotation benchmark. We also show that combining this sequence representation with protein 3D structure information does not lead to performance improvement, hinting that three-dimensional structure is also potentially learned during the unsupervised pre-training. Availability Implementations of all used models can be found at https://github.com/stamakro/GCN-for-Structure-and-Function. Supplementary information Supplementary data are available at Bioinformatics online.

The power of universal contextualised protein embeddings in cross-species protein function prediction

“…We already showed that SeqVec embeddings achieve competitive performance when applied in the task of molecular function prediction [22]. We built upon this previous work to characterise SeqVec-based molecular function prediction.…”

“…Whereas the new generation techniques usually outperform the established BLAST baseline method, they also require vast amounts of protein data which is not always comprehensive. Therefore, the most recent approaches often turn to automatic representation learning by which a complex model (often a neural network) learns some abstract features of a protein sequence which contain useful information for a consequent computational function prediction task [20][21][22]. and (ii) how the effects of these side chains vary across environmental context (i.e.…”

“…In this paper, we build upon previous work [22,23] and use the SeqVec embeddings to examine the reliability of predicting functions between proteins of evolutionary distant species. Till date, making predictions over large evolutionary distances is difficult as the function of a protein is determined by the context of its species [15].…”

Representation learning applications in biological sequence analysis

“…However, it was better than ProtVec [31] which is a context-independent model. In some tasks, such as protein function prediction, it outperformed one-hot encoding of k-mer-based embeddings and showed the competitive results obtained using ELMo [75].…”

Automatic Gene Function Prediction in the 2020’s

“…It can be further enhanced by unsupervised pre-training, where a generic protein representation is learned [83][84][85][86] from all available sequences. This representation can then be used to predict GO terms [87]. But end-to-end training is also possible, where the weights of the unsupervised feature extractor are also fine-tuned to create an ontology-specific feature representation designed for predicting GO terms from that ontology.…”