An Introduction to Variational Autoencoders

Kingma, Diederik P.; Welling, Max

doi:10.1561/2200000056

Cited by 1,340 publications

(751 citation statements)

References 78 publications

Supporting

Mentioning

603

Contrasting

Order By: Relevance

“…A good introduction can be found in Ref. [23]. In short, a VAE consists of a probabilistic encoder q(z|σ) and a probabilistic decoder p(σ|z), converting the sequence into a continuous multidimensional latent variable z and back.…”

Section: B Variational Auto-encodermentioning

confidence: 99%

On generative models of T-cell receptor sequences

Isacchini

Sethna

Elhanati

et al. 2019

Preprint

View full text Add to dashboard Cite

T-cell receptors (TCR) are key proteins of the adaptive immune system, generated randomly in each individual, whose diversity underlies our ability to recognize infections and malignancies. Modeling the distribution of TCR sequences is of key importance for immunology and medical applications. Here, we compare two inference methods trained on high-throughput sequencing data: a knowledge-guided approach, which accounts for the details of sequence generation, supplemented by a physics-inspired model of selection; and a knowledge-free Variational Auto-Encoder based on deep artificial neural networks. We show that the knowledge-guided model outperforms the deep network approach at predicting TCR probabilities, while being more interpretable, at a lower computational cost.

show abstract

Section: B Variational Auto-encodermentioning

confidence: 99%

On generative models of T-cell receptor sequences

Isacchini

Sethna

Elhanati

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…The dense layers are followed by two layers representing the mean and log variance of a normal distribution. This allows to model hidden influence factors in a way that emphasizes the mean while retaining variation, rather than reducing the networks loss by outputting the mean 19,20 . Furthermore, a constraint is placed on these two layers to minimize the Kullback-Leibler divergence to a Gaussian (0 + ; 1), where s is a location shift parameter:…”

Section: Vae Encodermentioning

confidence: 99%

“…Variational autoencoders (VAE) estimating the underlying probability distribution of the input data 19,20 have been successfully used in scRNA-Seq, with a special focus on the latent variable space for dimension reduction and clustering 21,22 . It is worth noting that the two previously mentioned algorithms are not mutually exclusive and concepts may be combined for specific applications (NMF 12 and AE 12,17 ), becoming more practical if the latent variable space takes specific cell type classes into account across training of the network model.…”

mentioning

confidence: 99%

Gene set inference from single-cell sequencing data using a hybrid of matrix factorization and variational autoencoders

Lukassen

Ten

Eils

et al. 2019

Preprint

View full text Add to dashboard Cite

Recent advances in single-cell RNA sequencing (scRNA-Seq) have driven the simultaneous measurement of the expression of 1,000s of genes in 1,000s of single cells. These growing data sets allow us to model gene sets in biological networks at an unprecedented level of detail, in spite of heterogenous cell populations. Here, we propose an unsupervised deep neural network model that is a hybrid of matrix factorization and conditional variational autoencoders (CVA), which utilizes weights as matrix factorizations to obtain gene sets, while class-specific inputs to the latent variable space facilitate a plausible identification of cell types. This artificial neural network model seamlessly integrates functional gene set inference, experimental batch effect correction, and static gene identification, which we conceptually prove here for three single-cell RNA-Seq datasets and suggest for future single-cell-gene analytics.

show abstract

“…We will adopt a probabilistic framework for latent-variable modeling of data [7], where a generative model p θ (x, z) for data x and latent variables z is assumed:…”

Section: Introductionmentioning

confidence: 99%

“…In principle, this could be used to form disentangled representations. However, model posterior is often intractable [7], and variational methods are used to estimate it.…”

Section: Introductionmentioning

confidence: 99%

A Closer Look at Disentangling in β-VAE

Sikka

Zhong

Yin

et al. 2019

2019 53rd Asilomar Conference on Signals, Systems, and Computers

View full text Add to dashboard Cite

In many data analysis tasks, it is beneficial to learn representations where each dimension is statistically independent and thus disentangled from the others. If data generating factors are also statistically independent, disentangled representations can be formed by Bayesian inference of latent variables. We examine a generalization of the Variational Autoencoder (VAE), β-VAE, for learning such representations using variational inference. β-VAE enforces conditional independence of its bottleneck neurons controlled by its hyperparameter β. This condition is in general not compatible with the statistical independence of latents. By providing analytical and numerical arguments, we show that this incompatibility leads to a non-monotonic inference performance in β-VAE with a finite optimal β.

show abstract

An Introduction to Variational Autoencoders

Abstract: Variational autoencoders provide a principled framework for learning deep latent-variable models and corresponding inference models.

Cited by 1,340 publications

References 78 publications

On generative models of T-cell receptor sequences

On generative models of T-cell receptor sequences

Gene set inference from single-cell sequencing data using a hybrid of matrix factorization and variational autoencoders

A Closer Look at Disentangling in β-VAE

Contact Info

Product

Resources

About