Stability Oracle: A Structure-Based Graph-Transformer for Identifying Stabilizing Mutations

Abstract

Stabilizing proteins is a fundamental challenge in protein engineering and is almost always a prerequisite for the development of industrial and pharmaceutical biotechnologies. Here we present Stability Oracle: a structure-based graph-transformer framework that achieves state of the art performance on predicting the effect of a point mutation on a protein's thermodynamic stability ∆∆G). A strength of our model is its ability to identify stabilizing mutations, which often make up a small fraction of a protein's mutational landscape. Our framework introduces several data and machine learning innovations to overcome well-known challenges in data scarcity and bias, generalization, and computation time. Stability Oracle is first pre-trained on over 2M masked microenvironments and then fine-tuned using a novel data augmentation technique, Thermodynamic Permutations (TP), applied to an ~120K curated subset of the mega-scale cDNA display proteolysis dataset. This technique increases the original 120K mutations to over 2M thermodynamically valid ∆∆G measurements to generate the first structure training set that samples and balances all 380 mutation types. By using the masked microenvironment paradigm, Stability Oracle does not require a second mutant structure and instead uses amino acid structural embeddings to represent a mutation. This architectural design accelerates training and inference times: we can both train on 2M instances with just 119 structures and generate deep mutational scan (DMS) predictions from only the wildtype structure. We benchmark Stability Oracle with both experimental and alphafold structures of all proteins on T2837, a test set that aggregates the common test sets (SSym, S669, p53, and Myoglobin) with all additional experimental data from proteins with over a 30% sequence similarity overlap. We used TP augmented T2837 to evaluate performance for protein engineering: Stability-Oracle correctly identifies 48% of stabilizing mutations (∆∆G < -0.5 kcal/mol) and 74% of its stabilizing predictions are indeed stabilizing (18% and 8% of predictions were neutral and destabilizing, respectively). For a fair comparison between sequence and structure-based fine-tuned deep learning models, we build on the Prostata framework and fine-tune the sequence embeddings of ESM2 on our training set (Prostata-IFML). A head-to-head comparison demonstrates that Stability-Oracle outperforms Prostata-IFML on regression and classification even though the model is 548 times smaller and is pretrained with 4000 times fewer proteins, highlighting the advantages of learning from structures.

Transfer learning to leverage larger datasets for improved prediction of protein stability changes

“…First, the dynamic range of the proteolysis assay is limited to ~5 kcal/mol 19 , while experimental stability datasets such as our Fireprot dataset may include mutations with up to ±10 kcal/mol DDG°. This means models trained on Megascale have limited capability to predict large changes in stability, a property that we also observe in other recently published models utilizing the Megascale dataset 16,26 . Second, we found that surface mutations to cysteine were often observed to be highly stabilizing in the Megascale dataset, such that ThermoMPNN would heavily favor surface cysteine mutations unless omitted from the permitted residue options (Supplementary Fig.…”

“…As expected, Megascale-trained models outperformed those trained on the sparse, unbalanced Fireprot dataset, and that advantage was more dependent on the total number of mutations included (sparsity) than on the number of unique proteins in the training dataset. Another recent structurebased transfer learning method, Stability Oracle, observed similar performance boosts from both pre-training for sequence recovery and transfer learning using the larger, more robust Megascale dataset 26 .…”

Empirical validation of ProteinMPNN’s efficiency in enhancing protein fitness

“…Therefore, few methods can consistently perform well in different test sets ( Pucci et al, 2022 ; Benevenuta et al, 2023 ). Moreover, since most mutations lead to decreased fitness, datasets are dominated by harmful mutations, causing prediction models to overfit on predicting harmful mutations ( Montanucci et al, 2019 ; Benevenuta et al, 2023 ; Diaz et al, 2023 ). Presently, prediction models are primarily evaluated using Pearson correlation coefficients, classification accuracy, and error, with high accuracy often stemming from the prediction of the relatively high proportion of harmful mutations in the test set ( Diaz et al, 2023 ).…”

“…Despite some previous efforts to address these issues, including extending relevant datasets and achieving certain results ( Diaz et al, 2023 ), the severe lack of real data obtained from biological experiments continues to plague the community. In this context, the utilization of AI models with large-scale pretraining techniques could be one effective solution to this problem.…”

Machine Learning-Guided Protein Engineering

“…Alternatively, we can avoid the need for labels by masking a part of the input, e.g. , a residue in a protein sequence or structure, − and training a model that will predict the masked part. In other words, the original data ( e.g.…”