InfoVAE: Balancing Learning and Inference in Variational Autoencoders

Zhao, Shengjia; Song, Jiaming; Ermon, Stefano

doi:10.1609/aaai.v33i01.33015885

Cited by 198 publications

(238 citation statements)

References 3 publications

Supporting

Mentioning

219

Contrasting

Order By: Relevance

“…In the case of integrating breast-cancer data we found that the choice of an appropriate regularisation when training the autoencoders is imperative. Our results show that the integrative VAEs yield better (and more disentangled) representations when MMD is employed, which also corresponds to findings from other studies (Zhao et al, 2017;Chen et al, 2018). Moreover, we found that giving a moderately large weight to this regularisation term further improves the quality of the learned representations.…”

Section: Discussionsupporting

confidence: 92%

“…In response, Higgins et al (2017) control the influence of the disentanglement factor using a parameter β. Moreover, some approaches have experimented with different regularisation terms, such as the InfoVAE (Zhao et al, 2017), where Maximum Mean Discrepancy (MMD) is employed as an alternative to KL divergence. MMD (Gretton et al, 2007) is based on the concept that two distributions are identical if, and only if, all their moments are identical.…”

Section: Variational Autoencoders For Cancer Data Integrationmentioning

confidence: 99%

See 1 more Smart Citation

Variational autoencoders for cancer data integration: design principles and computational practice

Simidjievski

Bodnar

Tariq

et al. 2019

Preprint

View full text Add to dashboard Cite

International initiatives such as the Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) are collecting multiple data sets at different genome-scales with the aim to identify novel cancer bio-markers and predict patient survival. To analyse such data, several machine learning, bioinformatics and statistical methods have been applied, among them neural networks such as autoencoders. Although these models provide a good statistical learning framework to analyse multi-omic and/or clinical data, there is a distinct lack of work on how to integrate diverse patient data and identify the optimal design best suited to the available data.In this paper, we investigate several autoencoder architectures that integrate a variety of cancer patient data types (e.g., multi-omics and clinical data). We perform extensive analyses of these approaches and provide a clear methodological and computational framework for designing systems that enable clinicians to investigate cancer traits and translate the results into clinical applications. We demonstrate how these networks can be designed, built and, in particular, applied to tasks of integrative analyses of heterogeneous breast cancer data. The results show that these approaches yield relevant data representations that, in turn, lead to accurate and stable diagnosis.

show abstract

Section: Discussionsupporting

confidence: 92%

Section: Variational Autoencoders For Cancer Data Integrationmentioning

confidence: 99%

Variational autoencoders for cancer data integration: design principles and computational practice

Simidjievski

Bodnar

Tariq

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…(iii) The loss L v is a term calculated using the Maximum Mean Discrepancy (MMD) [25], as explained in the following; notice that the idea of combining VAE and MMD was used for the first time in [26], where the authors proved that infoVAE (VAE using MMD) is fast to train, stable and leads to a better learning of the features if compared to the traditional evidence lower bound (ELBO) [27] criterion used in VAEs. The basic idea of MMD is that two distributions are identical if and only if their moments are the same.…”

Section: Figurementioning

confidence: 99%

DeepRICH: learning deeply Cherenkov detectors

Fanelli

Pomponi

2020

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

Imaging Cherenkov detectors are largely used for particle identification (PID) in nuclear and particle physics experiments, where developing fast reconstruction algorithms is becoming of paramount importance to allow for near real time calibration and data quality control, as well as to speed up offline analysis of large amount of data.In this paper we present DeepRICH, a novel deep learning algorithm for fast reconstruction which can be applied to different imaging Cherenkov detectors. The core of our architecture is a generative model which leverages on a custom Variational Auto-encoder (VAE) combined to Maximum Mean Discrepancy (MMD), with a Convolutional Neural Network (CNN) extracting features from the space of the latent variables for classification.A thorough comparison with the simulation/reconstruction package FastDIRC is discussed in the text. DeepRICH has the advantage to bypass low-level details needed to build a likelihood, allowing for a sensitive improvement in computation time at potentially the same reconstruction performance of other established reconstruction algorithms.In the conclusions, we address the implications and potentialities of this work, discussing possible future extensions and generalization.

show abstract

“…Perhaps the closest approach to ours, are those of learning latent representations for data reconstruction. In variational autoencoders (VAEs) [37], [38], [39], [40], a continuous latent representation space is learned from the inputs, that can then be used to reconstruct inputs or generate new data that follow the same distribution as the data in the training set. In [5], the authors present a new way of training VAEs to learn discrete latent space representations, which naturally leads to a compression algorithm, since continuous (or full-precision) inputs can be mapped to discrete latent representations typically using fewer bits.…”

Section: Decision Stumpsmentioning

confidence: 99%

On Distributed Quantization for Classification

Hanna

Ezzeldin

Tara

et al. 2020

IEEE J. Sel. Areas Inf. Theory

View full text Add to dashboard Cite

We consider the problem of distributed feature quantization, where the goal is to enable a pretrained classifier at a central node to carry out its classification on features that are gathered from distributed nodes through communication constrained channels. We propose the design of distributed quantization schemes specifically tailored to the classification task: unlike quantization schemes that help the central node reconstruct the original signal as accurately as possible, our focus is not reconstruction accuracy, but instead correct classification. Our work does not make any apriori distributional assumptions on the data, but instead uses training data for the quantizer design. Our main contributions include: we prove NP-hardness of finding optimal quantizers in the general case; we design an optimal scheme for a special case; we propose quantization algorithms, that leverage discrete neural representations and training data, and can be designed in polynomial-time for any number of features, any number of classes, and arbitrary division of features across the distributed nodes. We find that tailoring the quantizers to the classification task can offer significant savings: as compared to alternatives, we can achieve more than a factor of two reduction in terms of the number of bits communicated, for the same classification accuracy.

show abstract

InfoVAE: Balancing Learning and Inference in Variational Autoencoders

Cited by 198 publications

References 3 publications

Variational autoencoders for cancer data integration: design principles and computational practice

Variational autoencoders for cancer data integration: design principles and computational practice

DeepRICH: learning deeply Cherenkov detectors

On Distributed Quantization for Classification

Contact Info

Product

Resources

About