TransMEP: Transfer learning on large protein language models to predict mutation effects of proteins from a small known dataset

Abstract

Machine learning-guided optimization has become a driving force for recent improvements in protein engineering. In addition, new protein language models are learning the grammar of evolutionarily occurring sequences at large scales. This work combines both approaches to make predictions about mutational effects that support protein engineering. To this end, an easy-to-use software tool called TransMEP is developed using transfer learning by feature extraction with Gaussian process regression. A large collection of datasets is used to evaluate its quality, which scales with the size of the training set, and to show its improvements over previous fine-tuning approaches. Wet-lab studies are simulated to evaluate the use of mutation effect prediction models for protein engineering. This showed that TransMEP finds the best performing mutants with a limited study budget by considering the trade-off between exploration and exploitation.

Biophysics-based protein language models for protein engineering

“…PLMs are an important component in many existing methods for low-N protein engineering. They have been used to extract protein sequence representations [3,[74][75][76], for finetuning on the low-N function data [76][77][78], and to generate auxiliary training data in more complex models [78][79][80]. Other computational strategies for addressing the low-N problem include Gaussian processes [75,81,82], augmenting regression models with sequence-based [15,83] or structure-based [84] scores, custom protein 10/45 representations that can produce pretraining data [85], representations of proteins' 3D shape [86], meta learning [87], and contrastive finetuning [88].…”