Assessing the performance of protein regression models

Richard, Michael; Kæstel-Hansen, Jacob; Groth, Paul; Bartels, Simon; Salomon, Jesper; Tian, Pengfei; Hatzakis, Nikos S.; Boomsma, Wouter

doi:10.1101/2023.06.18.545472

Cited by 4 publications

(7 citation statements)

References 60 publications

(89 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regime extrapolation tests a model’s ability to predict how mutations combine by training on single amino acid substitutions and predicting the effects of multiple substitutions [18, 19, 21, 22] (Figs. 3c and S4).…”

Section: Resultsmentioning

confidence: 99%

“…We found METL-Local and ESM-2 displayed the strongest performance for mutation extrapolation, achieving an average Spearman correlation of 0.77 across datasets. Position extrapolation evaluates a model's ability to generalize across sequence positions and make predictions for amino acid substitutions at sites that do not vary in the training data [17][18][19] (Fig. 3b).…”

Section: Generalization Abilities Of Biophysics-based Protein Languag...mentioning

confidence: 99%

See 1 more Smart Citation

Biophysics-based protein language models for protein engineering

Gelman,

Johnson,

Freschlin

et al. 2024

Preprint

View full text Add to dashboard Cite

Protein language models trained on evolutionary data have emerged as powerful tools for predictive problems involving protein sequence, structure, and function. However, these models overlook decades of research into biophysical factors governing protein function. We propose Mutational Effect Transfer Learning (METL), a protein language model framework that unites advanced machine learning and biophysical modeling. Using the METL framework, we pretrain transformer-based neural networks on biophysical simulation data to capture fundamental relationships between protein sequence, structure, and energetics. We finetune METL on experimental sequence-function data to harness these biophysical signals and apply them when predicting protein properties like thermostability, catalytic activity, and fluorescence. METL excels in challenging protein engineering tasks like generalizing from small training sets and position extrapolation, although existing methods that train on evolutionary signals remain powerful for many types of experimental assays. We demonstrate METL's ability to design functional green fluorescent protein variants when trained on only 64 examples, showcasing the potential of biophysics-based protein language models for protein engineering.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Generalization Abilities Of Biophysics-based Protein Languag...mentioning

confidence: 99%

Biophysics-based protein language models for protein engineering

Gelman,

Johnson,

Freschlin

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…As an alternative, learned embeddings can be extracted from PLMs, such as those mentioned above. While these representations can offer performance boosts for certain tasks, they have not yet offered significant performance boosts compared to simple sequence encodings for supervised fitness prediction in MLDE or relevant protein engineering benchmarks such as predicting multimutant fitness from the fitness effects of single mutations. , Fine-tuning and semisupervised learning are other strategies to augment model performance when only a small amount of labeled data is available; this has shown initial promise but should be explored further . Additional benchmarks are needed to evaluate whether learned embeddings are more effective for ML-assisted protein fitness prediction.…”

Section: Navigating Protein Fitness Landscapes Using Machine Learningmentioning

confidence: 99%

Opportunities and Challenges for Machine Learning-Assisted Enzyme Engineering

Yang,

Li,

Arnold

2024

ACS Cent. Sci.

View full text Add to dashboard Cite

Enzymes can be engineered at the level of their amino acid sequences to optimize key properties such as expression, stability, substrate range, and catalytic efficiencyor even to unlock new catalytic activities not found in nature. Because the search space of possible proteins is vast, enzyme engineering usually involves discovering an enzyme starting point that has some level of the desired activity followed by directed evolution to improve its “fitness” for a desired application. Recently, machine learning (ML) has emerged as a powerful tool to complement this empirical process. ML models can contribute to (1) starting point discovery by functional annotation of known protein sequences or generating novel protein sequences with desired functions and (2) navigating protein fitness landscapes for fitness optimization by learning mappings between protein sequences and their associated fitness values. In this Outlook, we explain how ML complements enzyme engineering and discuss its future potential to unlock improved engineering outcomes.

show abstract

“…Although OHE offers no protein information aside from the amino acid identities, it is used extensively as a fast and effective method for converting biological information into numerical vectors (Elabd et al, 2020;Goldman et al, 2022;Greenhalgh et al, 2021;Hsu et al, 2022;Michael et al, 2023;Raimondi et al, 2019;M. Yang et al, 2018).…”

Section: Fixed Sequence Representationsmentioning

confidence: 99%

Protein Representations: Encoding Biological Information for Machine Learning in Biocatalysis

Harding-Larsen,

Funk,

Madsen

et al. 2024

Preprint

View full text Add to dashboard Cite

Enzymes offer a more environmentally friendly and low-impact solution to conventional chemistry, but they often require additional engineering for industrial settings, an endeavor that is challenging and laborious. To address this issue, the power of machine learning can be harnessed to produce predictive models that facilitate in silico study and engineering of novel enzymatic properties. However, the conversion from the biological domain to the computational realm requires special attention to ensure the training of accurate and precise models. In this review, we examine the critical step of encoding protein information to numeric representations for use in machine learning. We selected the most important approaches for encoding the three distinct biological protein representations — primary sequence, 3D structure, and dynamics — to explore their requirements for employment and inherent biases. Combined representations of proteins and substrates are also introduced as emergent tools in biocatalysis. We propose the division of fixed representations, a collection of rule-based encoding strategies, and learned representations extracted from the latent spaces of large neural networks. To select the most suitable protein representation, we propose two main factors governing this choice. The first one is the model setup, being influenced by the size of the training dataset and the choice of architecture. The second factor is the model objectives, concerning the assayed property, the difference between wild-type models and mutant predictors, and requirements for explainability. This review is aimed at serving as a source of information and guidance for properly representing enzymes in future machine learning models for biocatalysis.

show abstract

Assessing the performance of protein regression models

Cited by 4 publications

References 60 publications

Biophysics-based protein language models for protein engineering

Biophysics-based protein language models for protein engineering

Opportunities and Challenges for Machine Learning-Assisted Enzyme Engineering

Protein Representations: Encoding Biological Information for Machine Learning in Biocatalysis

Contact Info

Product

Resources

About