MQAPRank: improved global protein model quality assessment by learning-to-rank

Abstract

BackgroundProtein structure prediction has achieved a lot of progress during the last few decades and a greater number of models for a certain sequence can be predicted. Consequently, assessing the qualities of predicted protein models in perspective is one of the key components of successful protein structure prediction. Over the past years, a number of methods have been developed to address this issue, which could be roughly divided into three categories: single methods, quasi-single methods and clustering (or consensus) methods. Although these methods achieve much success at different levels, accurate protein model quality assessment is still an open problem.ResultsHere, we present the MQAPRank, a global protein model quality assessment program based on learning-to-rank. The MQAPRank first sorts the decoy models by using single method based on learning-to-rank algorithm to indicate their relative qualities for the target protein. And then it takes the first five models as references to predict the qualities of other models by using average GDT_TS scores between reference models and other models. Benchmarked on CASP11 and 3DRobot datasets, the MQAPRank achieved better performances than other leading protein model quality assessment methods. Recently, the MQAPRank participated in the CASP12 under the group name FDUBio and achieved the state-of-the-art performances.ConclusionsThe MQAPRank provides a convenient and powerful tool for protein model quality assessment with the state-of-the-art performances, it is useful for protein structure prediction and model quality assessment usages.

T4SE-XGB: Interpretable Sequence-Based Prediction of Type IV Secreted Effectors Using eXtreme Gradient Boosting Algorithm

“…In order to evaluate the overall predictive performance of different classification models, we used metrics such as Sensitivity (SE), Specificity (SP), Precision (PRE), Accuracy (ACC), F-score, and Matthew's correlation coefficient (MCC) to evaluate the model. They have been widely used in previous studies (Jing and Dong, 2017 ; Hu et al, 2018 ; Zhao et al, 2018a , b ; Al-Ajlan and El Allali, 2019 ; Chu et al, 2019 ; Lin et al, 2019 ; Manavalan et al, 2019 ; Zhang et al, 2019a , b ; Zhu X. et al, 2019 ; Cheng et al, 2020 ; Hasan et al, 2020 ; Liu et al, 2020 ; Yue et al, 2020 ; Zhang Y.-F. et al, 2020 ), with a higher value indicating better performances. The performance metrics can be defined as follows:…”

T4SE-XGB: interpretable sequence-based prediction of type IV secreted effectors using eXtreme gradient boosting algorithm

“…In order to evaluate the overall predictive performance of different classification models, we used metrics such as Sensitivity (SE), Specificity (SP), Precision (PRE), Accuracy (ACC), F-score and Matthew's correlation coefficient (MCC) to evaluate the model. They have been widely used in previous studies (Al-Ajlan and El Allali, 2019; Cheng, et al, 2020;Chu, et al, 2019;Hasan, et al, 2020;Jing and Dong, 2017;Lin, et al, 2019;Liu, et al, 2020;Manavalan, et al, 2019;Yue, et al, 2020;, with a higher value indicating better performances. The performance metrics can be defined as follows:…”

Smooth orientation-dependent scoring function for coarse-grained protein quality assessment

“…Thus, all the training CASP [5][6][7][8][9][10] data is used for both training and validation. However, the remaining CASP [11][12] datasets are not involved in this process and are left for the final evaluation. As a result of the described process, the regularization parameters were set to be α = 5, β = 50.…”

“…We also repeated a similar experiment using the CASP12 Stage1 and Stage2 data. For this experiment, the SBROD function was trained on CASP [5][6][7][8][9][10][11] data augmented with the generated NMA-based decoy models, and more recent methods were added for the comparison (Section B in Supplementary Information provides details on those). Tables 2 list the results on the original CASP12 server submissions, and Tables 3 list the results for the CASP12 data preprocessed with side-chains repacking.…”

“…In addition, single-model QA methods are proved to achieve better performance compared to consensus-model QA methods on unbalanced protein model pools and in cases where protein models within assessed pool are very similar (21). In addition to these two main types of QA methods, techniques combining both ideas have also been proposed (11,18), referred to as quasi-single model QA methods. Among recently proposed single-model QA methods, there are generally two main approaches to design a scoring function: physics-based and knowledge-based (data-driven) approaches (7,17).…”