Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoders

Abstract

Bayesian optimization is a gold standard for query-efficient continuous optimization. However, its adoption for drug and antibody sequence design has been hindered by the discrete, highdimensional nature of the decision variables. We develop a new approach (LaMBO) which jointly trains a denoising autoencoder with a discriminative multi-task Gaussian process head, enabling gradient-based optimization of multi-objective acquisition functions in the latent space of the autoencoder. These acquisition functions allow LaMBO to balance the explore-exploit trade-off over multiple design rounds, and to balance objective tradeoffs by optimizing sequences at many different points on the Pareto frontier. We evaluate LaMBO on a small-molecule task based on the ZINC dataset and introduce a new large-molecule task targeting fluorescent proteins. In our experiments, LaMBO outperforms genetic optimizers and does not require a large pretraining corpus, demonstrating that Bayesian optimization is practical and effective for biological sequence design.

Generalizing Bayesian Optimization with Decision-theoretic Entropies

“…Another potential direction is to incorporate cost-aware Bayesian optimization techniques into the EHIG framework [29,50,3]. We also wish to study how the proposed EHIG framework could be applied in practice to solve various problems in the sciences, including experimental physics [14,32,8], drug discovery [43,27,20], and materials design [31,46]. Finally, we wish to study in further detail how the EHIG acquisition function could be implemented for Bayesian decision making with other probabilistic models beyond Gaussian processes [42,10,9,34].…”

Accelerating Biocatalysis Discovery with Machine Learning: A Paradigm Shift in Enzyme Engineering, Discovery, and Design

“…Bayesian optimization is highly effective in striking this balance while being able to also incorporate prior knowledge about the problem, constraints to the search space, and the ability to optimize multiple objectives simultaneously. This has successfully been used in chemical synthesis engineering to reduce the number of experiments required to find the optimal conditions for a reaction and can similarly be applied to protein engineering. ,,,, …”

“…Stanton et al 79 combined a denoising autoencoder (a model similar to a VAE) with a Gaussian process regressor that predicts protein properties. Using a multiobjective Bayesian optimization approach, they generated novel red-spectrum fluorescent proteins (RFPs) with improved predicted stability and solvent-accessible surface areas (SASAs).…”

“…This has successfully been used in chemical synthesis engineering to reduce the number of experiments required to find the optimal conditions for a reaction 82 and can similarly be applied to protein engineering. 83,79,84,72,85 Integrating the aforementioned methods into iterative design-build-test-learn cycles can help in continuously improving protein engineering efforts over rounds of engineering. Using Bayesian Optimization in both candidate selection and experimental design, these cycles enable the systematic exploration of the protein sequence space and facilitate the discovery of novel protein variants with desired properties.…”

Now What Sequence? Pre-trained Ensembles for Bayesian Optimization of Protein Sequences

“…Figure S1 shows optimization of latent space has continuous improvement, but after decoding to an actual sequence and evaluating g( u) there is a plateau. This may be unique to UniRep or because or be task specific because other recent work has successfully used latent spaces for BayesOpt 68,69 and work on improving extrapolation from latent spaces. 70 Nevertheless, avoiding latent space negates this potential agreement problem between u and x.…”