Linear-Scaling Kernels for Protein Sequences and Small Molecules Outperform Deep Learning While Providing Uncertainty Quantitation and Improved Interpretability

Abstract

Gaussian process (GP) is a Bayesian model which provides several advantages for regression tasks in machine learning such as reliable quantitation of uncertainty and improved interpretability. Their adoption has been precluded by their excessive computational cost and by the difficulty in adapting them for analyzing sequences (e.g., amino acid sequences) and graphs (e.g., small molecules). In this study, we introduce a group of random feature-approximated kernels for sequences and graphs that exhibit linear scaling with both the size of the training set and the size of the sequences or graphs. We incorporate these new kernels into our new Python library for GP regression, xGPR, and develop an efficient and scalable algorithm for fitting GPs equipped with these kernels to large datasets. We compare the performance of xGPR on 17 different benchmarks with both standard and state-of-the-art deep learning models and find that GP regression achieves highly competitive accuracy for these tasks while providing with well-calibrated uncertainty quantitation and improved interpretability. Finally, in a simple experiment, we illustrate how xGPR may be used as part of an active learning strategy to engineer a protein with a desired property in an automated way without human intervention.

For antibody sequence generative modeling, mixture models may be all you need

“…The good results achieved with a simple model are reminiscent of recent studies where linear models or approximate Gaussian processes achieved performance competitive with fine-tuned LLMs or other deep learning models on fitness landscapes( Faure et al 2023 , Parkinson and Wang 2023 ). These results suggest it is important to use simple models as a baseline for protein engineering tasks.…”

For antibody sequence generative modeling, mixture models may be all you need

“…The good results achieved with a simple model are reminiscent of recent studies where linear models or approximate Gaussian processes achieved performance competitive with fine-tuned LLMs or other deep learning models on fitness landscapes 23,24 . These results suggest it is important to use simple models as a baseline for protein engineering tasks.…”

ChatGPT Generated Content and Similarity Index in Chemistry

“…The generated content can encompass a wide range of topics, such as organic chemistry, inorganic chemistry, analytical chemistry, physical chemistry, biochemistry, and other related areas. Several papers have already been published on chemistry and ChatGPT, such as drug discovery, teaching learning, computational chemistry, etc. − ChatGPT can be used to provide quick and accessible information about various aspects of chemistry and it may be a valuable tool for researchers, students, and professionals . Besides, ChatGPT can explain chemical concepts in an easier language, help students understand complex topics better, and potentially aid in problem-solving .…”