Auto-Encoding Variational Bayes

Abstract

How can we perform efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distributions, and large datasets? We introduce a stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case. Our contributions is two-fold. First, we show that a reparameterization of the variational lower bound yields a lower bound estimator that can be straightforwardly optimized using standard stochastic gradient methods. Second, we show that for i.i.d. datasets with continuous latent variables per datapoint, posterior inference can be made especially efficient by fitting an approximate inference model (also called a recognition model) to the intractable posterior using the proposed lower bound estimator. Theoretical advantages are reflected in experimental results.

Data-driven discovery of cardiolipin-selective small molecules by computational active learning

“…This approach renders our design space amenable to the construction of low-dimensional and robust surrogate models and the use of off-the-shelf Bayesian optimization algorithms. 60,61 We learn an appropriate latent space embedding in a data-driven fashion by training a regularized autoencoder (RAE), 62 a deterministic adaptation of the variational autoencoder (VAE) architecture, 63 over the corpus of 124,327 CG molecular graphs (Fig. 2b).…”

Unsupervised Learning in Drug Design from Self-Organization to Deep Chemistry

“…Born and Manica [ 42 ] predicted that multimodal deep learning chemistry using disparate sources to generate molecules would be the next challenge in DL in the near future. The Variational Autoencoder (VAE) method [ 44 ] was developed as an algorithm to learn continuous molecular representations. This method was used by Gomez-Bombarelli et al for an automatic chemical design using a data-driven continuous representation of molecules.…”

Functional Magnetic Resonance Imaging data augmentation through conditional ICA

“…On fMRI data these methods do not make much sense as brain images are not invariant to such transformations. More advanced techniques [29] are based on generative models such as GANs or variational auto-encoders [12]. Although GAN-based method are powerful they are slow and difficult to train [2].…”

“…Our method is not an adversarial procedure. However it relates to other powerful generative models such as variational auto-encoders [12] with which it shares strong similarities. Indeed the analog of the encoding function in the variational auto-encoder is given by e(x) = Λ − 1 2 q(W rest x) in our model and the analog to the decoding function in the variational auto-encoder is given by…”