Mixed-curvature Variational Autoencoders

Skopek, Ondrej; Ganea, Octavian-Eugen; Bécigneul, Gary

doi:10.48550/arxiv.1911.08411

Cited by 9 publications

(21 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are many isotropy invariant potentials on the sphere. Natural choices include the uniform density (which is invariant to all rotations) and the wrapped distribution with the center at v [29,35]. For our experiments, we use the uniform density.…”

Section: Invariant Potential Parameterizationmentioning

confidence: 99%

Equivariant Manifold Flows

Katsman¹,

Lou²,

Lim³

et al. 2021

Preprint

View full text Add to dashboard Cite

Tractably modelling distributions over manifolds has long been an important goal in the natural sciences. Recent work has focused on developing general machine learning models to learn such distributions. However, for many applications these distributions must respect manifold symmetries-a trait which most previous models disregard. In this paper, we lay the theoretical foundations for learning symmetry-invariant distributions on arbitrary manifolds via equivariant manifold flows. We demonstrate the utility of our approach by using it to learn gauge invariant densities over SU (n) in the context of quantum field theory.* indicates equal contribution 1 SU (n) denotes the special unitary group SU (n) = {X ∈ C n×n | X * X = I, det(X) = 1}.

show abstract

Section: Invariant Potential Parameterizationmentioning

confidence: 99%

Equivariant Manifold Flows

Katsman¹,

Lou²,

Lim³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…A mismatch between the latent prior distribution of a VAE and the data manifold structure can lead to over regularization and poor data representation. Other models incorporate more complex latent structures such as hyperspheres [12], tori [13,14], and hyperboloids [15,16,17]. These models have demonstrated their suitability for certain datasets by choosing a prior that is the best match out of a predefined set of candidates.…”

Section: Table 1: Comparison Of Vae Techniquesmentioning

confidence: 99%

Variational Autoencoder with Learned Latent Structure

Connor,

Canal,

Rozell

2020

Preprint

View full text Add to dashboard Cite

The manifold hypothesis states that high-dimensional data can be modeled as lying on or near a low-dimensional, nonlinear manifold. Variational Autoencoders (VAEs) approximate this manifold by learning mappings from low-dimensional latent vectors to high-dimensional data while encouraging a global structure in the latent space through the use of a specified prior distribution. When this prior does not match the structure of the true data manifold, it can lead to a less accurate model of the data. To resolve this mismatch, we introduce the Variational Autoencoder with Learned Latent Structure (VAELLS) which incorporates a learnable manifold model into the latent space of a VAE. This enables us to learn the nonlinear manifold structure from the data and use that structure to define a prior in the latent space. The integration of a latent manifold model not only ensures that our prior is well-matched to the data, but also allows us to define generative transformation paths in the latent space and describe class manifolds by transformations stemming from examples of each class. We validate our model on examples with known latent structure and also demonstrate its capabilities on a real-world dataset. 1

show abstract

“…However, it has recently been shown that, for datasets with hierarchical structure, there are clear advantages to instead using latent variables that have support on the Poincaré disk. This observation has motivated the Poincaré VAE (PVAE, Mathieu et al (2019)), the Poincaré Wasserstein auto-encoder (Ovinnikov, 2019), Adversarial PVAE (Dai et al, 2020) and Mixed-curvture VAE (Skopek et al, 2019) among others.…”

Section: Poincaré Diskmentioning

confidence: 99%

Density estimation and modeling on symmetric spaces

Li¹,

Lu²,

Chevalier³

et al. 2020

Preprint

View full text Add to dashboard Cite

In many applications, data and/or parameters are supported on non-Euclidean manifolds. It is important to take into account the geometric structure of manifolds in statistical analysis to avoid misleading results. Although there has been a considerable focus on simple and specific manifolds, there is a lack of general and easy-to-implement statistical methods for density estimation and modeling on manifolds. In this article, we consider a very broad class of manifolds: non-compact Riemannian symmetric spaces. For this class, we provide a very general mathematical result for easily calculating volume changes of the exponential and logarithm map between the tangent space and the manifold. This allows one to define statistical models on the tangent space, push these models forward onto the manifold, and easily calculate induced distributions by Jacobians. To illustrate the statistical utility of this theoretical result, we provide a general method to construct distributions on symmetric spaces. In particular, we define the log-Gaussian distribution as an analogue of the multivariate Gaussian distribution in Euclidean space. With these new kernels on symmetric spaces, we also consider the problem of density estimation. Our proposed approach can use any existing density estimation approach designed for Euclidean spaces and push it forward to the manifold with an easy-to-calculate adjustment. We provide theorems showing that the induced density estimators on the manifold inherit the statistical optimality properties of the parent Euclidean density estimator; this holds for both frequentist and Bayesian nonparametric methods. We illustrate the theory and practical utility of the proposed approach on the space of positive definite matrices.

show abstract

Mixed-curvature Variational Autoencoders

Cited by 9 publications

References 20 publications

Equivariant Manifold Flows

Equivariant Manifold Flows

Variational Autoencoder with Learned Latent Structure

Density estimation and modeling on symmetric spaces

Contact Info

Product

Resources

About