Improved protein contact prediction using dimensional hybrid residual networks and singularity enhanced loss function

Abstract

Deep residual learning has shown great success in protein contact prediction. In this study, a new deep residual learning-based protein contact prediction model was developed. Comparing with previous models, a new type of residual block hybridizing 1D and 2D convolutions was designed to increase the effective receptive field of the residual network, and a new loss function emphasizing the easily misclassified residue pairs was proposed to enhance the model training. The developed protein contact prediction model referred to as DRN-1D2D was first evaluated on 105 CASP 11 targets, 76 CAMEO hard targets and 398 membrane proteins together with two in house-developed reference models based on either the standard 2D residual block or the traditional BCE loss function, from which we confirmed that that the dimensional hybrid residual block and the singularity enhanced loss function can both be employed to improve the model performance for protein contact prediction. DRN-1D2D was further evaluated on 39 CASP 13 and CASP 14 free modeling targets together with the two reference models and four state-of-the-art protein contact prediction models including DeepCov, DeepCon, RaptorX-Contact and TripleRes. The result shows that DRN-1D2D consistently achieved the best performance among all these models.

Improved inter-protein contact prediction using dimensional hybrid residual networks and protein language models

“…Seven independent models were trained separately, and the final prediction was the average of the predictions from the seven models. We used the singularity enhanced loss function proposed in (Si and Yan, 2021) 20 to calculate training loss and optimized the loss with AdamW optimizer with its default settings. During training, if the loss on the validation set did not drop within two epochs, the learning rate would decay to 0.1 of its original (the initial learning rate was 0.001).…”

“…The input features are then transformed by the dimensional hybrid residual network formed by dimensional hybrid residual blocks (1D2D blocks). A detailed description of the 1D2D block can be found in (Si and Yan, 2021) 20 . In this study, we increased the length of the 1D convolution kernels from 9 to 15, for we found using the longer 1D kernels slightly improved the model performance.…”

“…Seven independent models were trained separately, and the final prediction was the average of the predictions from the seven models. We used the singularity enhanced loss function proposed in (Si and Yan, 2021) 20 to calculate training loss and optimized the loss with AdamW optimizer with its default settings.…”

“…The structural information enriched features obtained from protein language models were further combined with traditional statistical features including position-specific scoring matrices (PSSM) and inter-protein coevolution to form the input features of our prediction model, which were then transformed by the residual network formed by dimensional hybrid residual blocks to predict inter-protein contacts. It was shown in our previous study that the application of the dimensional hybrid residual blocks can increase the effective receptive field of the residual network, and thus improve the model performance 20 . The developed model referred to as DRN-1D2D_Inter was extensively benchmarked on PPI dataset including both homomeric PPIs and heteromeric PPIs.…”

Improved inter-residue contact prediction via a hybrid generative model and dynamic loss function

“…Notably, DCA methods generally exhibit two major limitations. First, DCA-based methods have poor performance when the number of homologous sequences is lower than approximately 50 [21] , [22] . Second, these methods extract only linear relationships between pairs of residues [21] .…”