RFQAmodel: Random Forest Quality Assessment to identify a predicted protein structure in the correct fold

Abstract

While template-free protein structure prediction protocols now produce good quality models for many targets, modelling failure remains common. For these methods to be useful it is important that users can both choose the best model from the hundreds to thousands of models that are commonly generated for a target, and determine whether this model is likely to be correct. We have developed Random Forest Quality Assessment (RFQAmodel), which assesses whether models produced by a protein structure prediction pipeline have the correct fold. RFQAmodel uses a combination of existing quality assessment scores with two predicted contact map alignment scores.These alignment scores are able to identify correct models for targets that are not otherwise captured. Our classifier was trained on a large set of protein domains that are structurally diverse and evenly balanced in terms of protein features known to have an effect on modelling success, and then tested on a second set of 244 protein domains with a similar spread of properties. When models for each target in this second set were ranked according to the RFQAmodel score, the highest-ranking model had a high-confidence RFQAmodel score for 67 modelling targets, of which 52 had the correct fold. At the other end of the scale RFQAmodel correctly predicted that for 59 targets the highest-ranked model was incorrect. In comparisons to other methods we found that May 23, 2019 1/23 RFQAmodel is better able to identify correct models for targets where only a few of the models are correct. We found that RFQAmodel achieved a similar performance on the model sets for CASP12 and CASP13 free-modelling targets. Finally, by iteratively generating models and running RFQAmodel until a model is produced that is predicted to be correct with high confidence, we demonstrate how such a protocol can be used to focus computational efforts on difficult modelling targets. Introduction 1 Template-free protein structure prediction protocols routinely produce hundreds to 2 thousands of models for a given target [1]. Users need to be able to identify if a good 3 model exists in this ensemble. The final step in a typical structure prediction pipeline is 4 therefore to select a representative subset of five or fewer models as output [2]. This 5 model selection step is critical, and the community's ability to select good models is 6 assessed as part of the Critical Assessment of protein Structure Prediction (CASP) 7 experiments [3]. 8 Protocols for model quality assessment can be divided into three classes: 9 single-model methods, quasi-single model methods, and consensus methods [2]. 10 Single-model methods calculate a score for each model independently, and this score 11 does not take into account any of the other models generated for a particular target. 12 The objective function optimised during protein structure prediction can usually be 13 used as a single-model quality estimator, but better results have been reported if 14 different scores are used for modelling and ranking [2]. Examples of single...