Directed Weight Neural Networks for Protein Structure Representation Learning

Abstract

A protein performs biological functions by folding to a particular 3D structure. To accurately model the protein structures, both the overall geometric topology and local fine-grained relations between amino acids (e.g. side-chain torsion angles and inter-amino-acid orientations) should be carefully considered. In this work, we propose the Directed Weight Neural Network for better capturing geometric relations among different amino acids. Extending a single weight from a scalar to a 3D directed vector, our new framework supports a rich set of geometric operations on both classical and SO(3)-representation features, on top of which we construct a perceptron unit for processing amino-acid information. In addition, we introduce an equivariant message passing paradigm on proteins for plugging the directed weight perceptrons into existing Graph Neural Networks, showing superior versatility in maintaining SO(3)-equivariance at the global scale. Experiments show that our network has remarkably better expressiveness in representing geometric relations in comparison to classical neural networks and the (globally) equivariant networks. It also achieves state-of-the-art performance on various computational biology applications related to protein 3D structures. All codes and models will be published upon acceptance.

Learning Hierarchical Protein Representations via Complete 3D Graph Networks

“…Note that most existing approaches directly integrate backbone structural information into amino acid features. These methods firstly compute three backbone torsion angles ω 1 , ω 2 , and ω 3 [24,25,15,33] for each amino acid as shown in Fig. 2(c).…”

“…Learning protein representations is essential to a variety of tasks in protein engineering [7,52,61,63,15,51,40,35]. Existing methods for protein learning consider different kinds of protein information, including amino acid sequences [43,3,46,11,6], protein surfaces [14,53,8,49], and protein 3D structures [13,19,2,21,20,26,25,33,64]. Due to recent advances in protein structure prediction [48,27,57,1], structures of many proteins are becoming available with high accuracy.…”

“…However, grid-like protein data is extremely sparse, leading to expensive learning cost and unsatisfactory performance. Hence, recent studies use 3D GNNs to process protein data modeled as 3D protein graphs [21,20,26,24,64,33]. Based on the analysis in Sec.…”

“…We report the accuracies on the three test sets and the average of the three accuracy values in Table 1. The results for baseline methods are taken from original papers [21,49,64,33].…”

“…Importantly, there exist hierarchical relations among different levels. Existing methods for protein representation learning either ignore hierarchical relations within proteins [26,33,20,64], or suffer from excessive computational complexity [25,21]. In this work, Figure 1: Illustration of hierarchical representations for protein structures.…”