Emerging Convolutions for Generative Normalizing Flows

Hoogeboom, Emiel; Berg, Rianne van den; Welling, Max

doi:10.48550/arxiv.1901.11137

Cited by 10 publications

(18 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each of these steps can be decomposed into layers steps. Improvements for some of these layers where proposed in later articles (Kingma and Dhariwal [2018], Hoogeboom et al [2019]).…”

Section: Related Workmentioning

confidence: 99%

“…These can easily be inverted by inverting the kernel. Finally, Hoogeboom et al [2019] replaced this 1x1 convolution with the so-called emerging convolution. These have the same receptive convolution with a kernel of arbitrary size.…”

Section: Actnormmentioning

confidence: 99%

“…Ho et al [2019] proposed a flow-based model that is the non-autoregressive model for unconditional density estimation on standard image benchmarks Invertible 1×1 Convolution: Kingma and Dhariwal [2018] proposed the invertible 1×1 convolution replacing fixed permutation (Dinh et al [2017]) that reverses the ordering of the channels. Hoogeboom et al [2019] proposed normalizing flow method to do the inversion of 1×1 convolution with doing some padding on the kernel and two distinct autoregressive convolutions, which also provide a stable and flexible parameterization for invertible 1×1 convolutions.…”

Section: Actnormmentioning

confidence: 99%

“…In our proposed method, there is no need for the reformulation of standard convolutions. Hoogeboom et al [2019] proposed two different methods to produce the invertible convolutions : (1) Emerging Convolution and (2) Invertible Periodic Convolutions. Emerging requires two autoregressive convolutions to do a standard convolution, but our method requires only one convolution as compare to the method proposed by Hoogeboom et al [2019] and increase the flexibility of the invertible n×n convolution.…”

Section: Actnormmentioning

confidence: 99%

“…Emerging convolutions proposed a way of inverting convolutions with large window sizes Hoogeboom et al [2019]. The inverse is not a convolution and is computed by a linear equation system that can efficiently be solved using back substitution.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

CInC Flow: Characterizable Invertible 3x3 Convolution

Nagar,

Dufraisse,

Varma

2021

Preprint

View full text Add to dashboard Cite

Normalizing flows are an essential alternative to GANs for generative modelling, which can be optimized directly on the maximum likelihood of the dataset. They also allow computation of the exact latent vector corresponding to an image since they are composed of invertible transformations. However, the requirement of invertibility of the transformation prevents standard and expressive neural network models such as CNNs from being directly used. Emergent convolutions were proposed to construct an invertible 3×3 CNN layer using a pair of masked CNN layers, making them inefficient.We study conditions such that 3×3 CNNs are invertible, allowing them to construct expressive normalizing flows. We derive necessary and sufficient conditions on a padded CNN for it to be invertible. Our conditions for invertibility are simple, can easily be maintained during the training process. Since we require only a single CNN layer for every effective invertible CNN layer, our approach is more efficient than emerging convolutions. We also proposed a coupling method, Quad-coupling. We benchmark our approach and show similar performance results to emergent convolutions while improving the model's efficiency.

show abstract

“…Each of these steps can be decomposed into layers steps. Improvements for some of these layers where proposed in later articles (Kingma and Dhariwal [2018], Hoogeboom et al [2019]).…”

Section: Related Workmentioning

confidence: 99%

Section: Actnormmentioning

confidence: 99%

Section: Actnormmentioning

confidence: 99%

Section: Actnormmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

CInC Flow: Characterizable Invertible 3x3 Convolution

Nagar,

Dufraisse,

Varma

2021

Preprint

View full text Add to dashboard Cite

show abstract

Inferring the quantum density matrix with machine learning

Cranmer,

Golkar,

Pappadopulo

2019

Preprint

View full text Add to dashboard Cite

We introduce two methods for estimating the density matrix for a quantum system: Quantum Maximum Likelihood and Quantum Variational Inference. In these methods, we construct a variational family to model the density matrix of a mixed quantum state. We also introduce quantum flows, the quantum analog of normalizing flows, which can be used to increase the expressivity of this variational family. The eigenstates and eigenvalues of interest are then derived by optimizing an appropriate loss function. The approach is qualitatively different than traditional lattice techniques that rely on the time dependence of correlation functions that summarize the lattice configurations. The resulting estimate of the density matrix can then be used to evaluate the expectation of an arbitrary operator, which opens the door to new possibilities.

show abstract

Cubic-Spline Flows

Durkan,

Bekasov,

Murray

et al. 2019

Preprint

View full text Add to dashboard Cite

A normalizing flow models a complex probability density as an invertible transformation of a simple density. The invertibility means that we can evaluate densities and generate samples from a flow. In practice, autoregressive flow-based models are slow to invert, making either density estimation or sample generation slow. Flows based on coupling transforms are fast for both tasks, but have previously performed less well at density estimation than autoregressive flows. We stack a new coupling transform, based on monotonic cubic splines, with LU-decomposed linear layers. The resulting cubic-spline flow retains an exact onepass inverse, can be used to generate high-quality images, and closes the gap with autoregressive flows on a suite of density-estimation tasks.

show abstract

Emerging Convolutions for Generative Normalizing Flows

Cited by 10 publications

References 0 publications

CInC Flow: Characterizable Invertible 3x3 Convolution

CInC Flow: Characterizable Invertible 3x3 Convolution

Inferring the quantum density matrix with machine learning

Cubic-Spline Flows

Contact Info

Product

Resources

About