Role of simple descriptors and applicability domain in predicting change in protein thermostability

Abstract

The melting temperature (Tm) of a protein is the temperature at which half of the protein population is in a folded state. Therefore, Tm is a measure of the thermostability of a protein. Increasing the Tm of a protein is a critical goal in biotechnology and biomedicine. However, predicting the change in melting temperature (dTm) due to mutations at a single residue is difficult because it depends on an intricate balance of forces. Existing methods for predicting dTm have had similar levels of success using generally complex models. We find that training a machine learning model with a simple set of easy to calculate physicochemical descriptors describing the local environment of the mutation performed as well as more complicated machine learning models and is 2–6 orders of magnitude faster. Importantly, unlike in most previous publications, we perform a blind prospective test on our simple model by designing 96 variants of a protein not in the training set. Results from retrospective and prospective predictions reveal the limited applicability domain of each model. This study highlights the current deficiencies in the available dTm dataset and is a call to the community to systematically design a larger and more diverse experimental dataset of mutants to prospectively predict dTm with greater certainty.

Evaluating Protein Engineering Thermostability Prediction Tools Using an Independently Generated Dataset

“…This further validates that current algorithms have limited utility for proteins outside of those they were benchmarked on. This limitation hindered by an over-representation of protein families such as lysozyme, tryptophan synthase, and ribonuclease in curated datasets is often utilized in benchmarking . Thus, this highlights the importance of generating high-quality and diverse datasets of more proteins for evaluating and training new computational tools.…”

A critical review of five machine learning-based algorithms for predicting protein stability changes upon mutation

“…However, the performance of these algorithms has been challenged as several performance reviews failed to reproduce the high accuracies found by the authors [29,30]. Recently, in a blind prospective validation, none of 63 mutants predicted to be stabilizing were experimentally observed to be stabilizing [31].…”

Data set and fitting dependencies when estimating protein mutant stability: Toward simple, balanced, and interpretable models

“…75,76 The sensitivity to variations in the used wild-type structure is substantial, 72,75,[77][78][79] and overfitting to structure is suspected from the observation that in some cases, older PDB files used for training perform better than newer higher-resolution structures. 66 Recent studies 46,66,68,69,71,75,77,80,81 have attempted to address these biases, including the internal consistency in terms of hysteresis, 68,72,78 with recent data sets providing a basis for such benchmarking. 69 Specifically, there should be no hysteresis in the evaluation of ΔΔG 72 ; since the four-state cycle has been reduced to interpolation from a single folded structure, this hysteresis simplifies to a difference in the forward and backward calculation of ΔΔG, which is possible to compute for some proteins where mutant structures are available.…”