Revisiting Bayesian Autoencoders with MCMC

Abstract

Autoencoders gained popularity in the deep learning revolution given their ability to compress data and provide dimensionality reduction. Although prominent deep learning methods have been used to enhance autoencoders, the need to provide robust uncertainty quantification remains a challenge. This has been addressed with variational autoencoders so far. Bayesian inference via MCMC methods have faced limitations but recent advances with parallel computing and advanced proposal schemes that incorporate gradients have opened routes less travelled. In this paper, we present Bayesian autoencoders powered MCMC sampling implemented using parallel computing and Langevin gradient proposal scheme. Our proposed Bayesian autoencoder provides similar performance accuracy when compared to related methods from the literature, with the additional feature of robust uncertainty quantification in compressed datasets. This motivates further application of the Bayesian autoencoder framework for other deep learning models.

Bayesian autoencoders with uncertainty quantification: Towards trustworthy anomaly detection

“…Tran et al [9] observed that BAE-generated data have higher uncertainty than the reconstructed data. Chandra et al [26] studied the BAE with Markov Chain Monte Carlo (MCMC) sampling for dimensionality reduction and classification tasks.…”

SMOTified-GAN for class imbalanced pattern classification problems

“…A major limitation for MCMC sampling technique is high computational complexity for sampling from the posterior distribution [31,32]. There recently there has been much progress in MCMC sampling via the use of gradient-based proposals and parallel computing in Bayesian deep learning [33,34,35]. However, these have been mostly limited to model parameter (weights) uncertainty quantification rather than quantifying uncertainties in data or addressing class imbalanced problems.…”