Latent Normalizing Flows for Discrete Sequences

Ziegler, Zachary M.; Rush, Alexander M.

doi:10.48550/arxiv.1901.10548

Cited by 11 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Flow++ [19] uses the CDF of a mixture of logistic distributions as a monotonic transformation in coupling layers, but requires bisection search to compute an inverse, since a closed form is not available. Non-linear squared flow [59] adds an inverse-quadratic perturbation to an affine transformation in an autoregressive flow, which is invertible under certain restrictions of the parameterization. Computing this inverse requires solving a cubic polynomial, and the overall transform is less flexible than a monotonic rational-quadratic spline.…”

Section: Related Workmentioning

confidence: 99%

Neural Spline Flows

Durkan,

Bekasov,

Murray

et al. 2019

Preprint

115

View full text Add to dashboard Cite

A normalizing flow models a complex probability density as an invertible transformation of a simple base density. Flows based on either coupling or autoregressive transforms both offer exact density evaluation and sampling, but rely on the parameterization of an easily invertible elementwise transformation, whose choice determines the flexibility of these models. Building upon recent work, we propose a fully-differentiable module based on monotonic rational-quadratic splines, which enhances the flexibility of both coupling and autoregressive transforms while retaining analytic invertibility. We demonstrate that neural spline flows improve density estimation, variational inference, and generative modeling of images. * Equal contribution Preprint. Under review.

show abstract

Section: Related Workmentioning

confidence: 99%

Neural Spline Flows

Durkan,

Bekasov,

Murray

et al. 2019

Preprint

115

View full text Add to dashboard Cite

show abstract

“…Most state-of-the-art models are affine coupling-based NFs such as Glow (section 4.3) and variants of it (Section 4.4). Other methods such as Nonlinear squared, Continuous Mixed CDF, Spline, Neural Autoregressive, Sum-of-Square and Real-and-Discrete coupling flows [11,21,[34][35][36][37][38][39] exist but has not seen equal success.…”

Section: Coupling Flowsmentioning

confidence: 99%

Out-of-Distribution Detection of Melanoma using Normalizing Flows

Valiuddin¹,

Viviers²

2021

Preprint

View full text Add to dashboard Cite

Generative modelling has been a topic at the forefront of machine learning research for a substantial amount of time. With the recent success in the field of machine learning, especially in deep learning, there has been an increased interest in explainable and interpretable machine learning. The ability to model distributions and provide insight in the density estimation and exact data likelihood is an example of such a feature. Normalizing Flows (NFs), a relatively new research field of generative modelling, has received substantial attention since it is able to do exactly this at a relatively low cost whilst enabling competitive generative results. While the generative abilities of NFs are typically explored, we focus on exploring the data distribution modelling for Out-of-Distribution (OOD) detection. Using one of the state-of-the-art NF models, GLOW, we attempt to detect OOD examples in the ISIC dataset. We notice that this model under performs in conform related research. To improve the OOD detection, we explore the masking methods to inhibit co-adaptation of the coupling layers however find no substantial improvement. Furthermore, we utilize Wavelet Flow which uses wavelets that can filter particular frequency components, thus simplifying the modeling process to data-driven conditional wavelet coefficients instead of complete images. This enables us to efficiently model larger resolution images in the hopes that it would capture more relevant features for OOD. The paper that introduced Wavelet Flow mainly focuses on its ability of sampling high resolution images and did not treat OOD detection. We present the results and propose several ideas for improvement such as controlling frequency components, using different wavelets and using other state-of-the-art NF architectures.

show abstract

“…We first obtain the number of nodes N in the graph. For this the approach of Ziegler and Rush [41], Niu et al [27] is taken which samples from the empirical multinomial distribution of node sizes in the training data. Once N is fixed, we can sample matrix A ∈ R N ×N via Langevin dynamics on the energy function E θ .…”

Section: Graph Samplingmentioning

confidence: 99%

Adversarial Stein Training for Graph Energy Models

Shankar¹

2021

Preprint

View full text Add to dashboard Cite

Learning distributions over graph-structured data is a challenging task with many applications in biology and chemistry. In this work we use an energy-based model (EBM) based on multi-channel graph neural networks (GNN) to learn permutation invariant unnormalized density functions on graphs. Unlike standard EBM training methods our approach is to learn the model via minimizing adversarial stein discrepancy. Samples from the model can be obtained via Langevin dynamics based MCMC. We find that this approach achieves competitive results on graph generation compared to benchmark models.

show abstract

Latent Normalizing Flows for Discrete Sequences

Cited by 11 publications

References 0 publications

Neural Spline Flows

Neural Spline Flows

Out-of-Distribution Detection of Melanoma using Normalizing Flows

Adversarial Stein Training for Graph Energy Models

Contact Info

Product

Resources

About