VQ-Flows: Vector Quantized Local Normalizing Flows

Abstract

Normalizing flows provide an elegant approach to generative modeling that allows for efficient sampling and exact density evaluation of unknown data distributions. However, current techniques have significant limitations in their expressivity when the data distribution is supported on a lowdimensional manifold or has a non-trivial topology. We introduce a novel statistical framework for learning a mixture of local normalizing flows as "chart maps" over the data manifold. Our framework augments the expressivity of recent approaches while preserving the signature property of normalizing flows, that they admit exact density evaluation. We learn a suitable atlas of charts for the data manifold via a vector quantized autoencoder (VQ-AE) and the distributions over them using a conditional flow. We validate experimentally that our probabilistic framework enables existing approaches to better model data distributions over complex manifolds.

Neural Implicit Manifold Learning for Topology-Aware Generative Modelling

“…Awareness of the data manifold's topology may be necessary for downstream applications such as defending against adversarial examples [43] or out-of-distribution detection [15]. In the injective normalizing flows literature in particular, there has been interest in learning manifolds with multiple charts [44,76], which are certainly more expressive than using a single chart. Thus far, such approaches require ancillary models for inference, which can complicate density estimation, and must set the number of charts as a hyperparameter.…”