Cubic-Spline Flows

Durkan, Conor; Bekasov, Artur; Murray, Iain; Papamakarios, George

doi:10.48550/arxiv.1906.02145

Cited by 20 publications

(30 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most state-of-the-art models are affine coupling-based NFs such as Glow (section 4.3) and variants of it (Section 4.4). Other methods such as Nonlinear squared, Continuous Mixed CDF, Spline, Neural Autoregressive, Sum-of-Square and Real-and-Discrete coupling flows [11,21,[34][35][36][37][38][39] exist but has not seen equal success.…”

Section: Coupling Flowsmentioning

confidence: 99%

Out-of-Distribution Detection of Melanoma using Normalizing Flows

Valiuddin¹,

Viviers²

2021

Preprint

View full text Add to dashboard Cite

Generative modelling has been a topic at the forefront of machine learning research for a substantial amount of time. With the recent success in the field of machine learning, especially in deep learning, there has been an increased interest in explainable and interpretable machine learning. The ability to model distributions and provide insight in the density estimation and exact data likelihood is an example of such a feature. Normalizing Flows (NFs), a relatively new research field of generative modelling, has received substantial attention since it is able to do exactly this at a relatively low cost whilst enabling competitive generative results. While the generative abilities of NFs are typically explored, we focus on exploring the data distribution modelling for Out-of-Distribution (OOD) detection. Using one of the state-of-the-art NF models, GLOW, we attempt to detect OOD examples in the ISIC dataset. We notice that this model under performs in conform related research. To improve the OOD detection, we explore the masking methods to inhibit co-adaptation of the coupling layers however find no substantial improvement. Furthermore, we utilize Wavelet Flow which uses wavelets that can filter particular frequency components, thus simplifying the modeling process to data-driven conditional wavelet coefficients instead of complete images. This enables us to efficiently model larger resolution images in the hopes that it would capture more relevant features for OOD. The paper that introduced Wavelet Flow mainly focuses on its ability of sampling high resolution images and did not treat OOD detection. We present the results and propose several ideas for improvement such as controlling frequency components, using different wavelets and using other state-of-the-art NF architectures.

show abstract

Section: Coupling Flowsmentioning

confidence: 99%

Out-of-Distribution Detection of Melanoma using Normalizing Flows

Valiuddin¹,

Viviers²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In this regime, density estimation techniques based on neural networks are becoming more and more popular. One class of these neural density estimation techniques are normalizing flows (17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30)(31)(32), in which variables described by a simple base distribution p(u) such as a multivariate Gaussian are transformed through a parameterized invertible transformation x = g φ (u) that has a tractable Jacobian. The target density pg(x) is then given by the change-of-variables formula as a product of the base density and the determinant of the transformation's Jacobian.…”

Section: Frontiers Of Simulation-based Inferencementioning

confidence: 99%

The frontier of simulation-based inference

Cranmer,

Brehmer,

Louppe

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Splines have also been used as building blocks of normalizing flows: Müller et al (2018) suggested modelling a linear and quadratic spline as the integral of a univariate monotonic function for flow construction. Durkan et al (2019a) proposed a natural extension to the framework of neural importance sampling and also suggested modelling a coupling layer as a monotonic rational-quadratic spine (Durkan et al, 2019b), which can be implemented either with a coupling architecture RQ-NSF(C) or with autoregressive architecture RQ-NSF(AR).…”

Section: Related Workmentioning

confidence: 99%

Identifying through Flows for Recovering Latent Representations

Shen¹,

Hooi²,

Lee³

2019

Preprint

View full text Add to dashboard Cite

Identifiability, or recovery of the true latent representations from which the observed data originates, is a fundamental goal of representation learning. However, most deep generative models do not address the question of identifiability, and cannot recover the true latent sources that generate the observations. Recent work proposed identifiable generative modelling using variational autoencoders (iVAE) with a theory of identifiability. However, due to the intractablity of KL divergence between variational approximate posterior and the true posterior, iVAE has to maximize the evidence lower bound of the marginal likelihood, leading to suboptimal solutions in both theory and practice. In contrast, we propose an identifiable framework for estimating latent representations using a flow-based model (iFlow). Our approach directly maximizes the marginal likelihood, allowing for theoretical guarantees on identifiability, without the need for variational approximations. We derive its learning objective in analytical form, making it possible to train iFlow in an end-to-end manner. Simulations on synthetic data validate the correctness and effectiveness of our proposed method and demonstrate its practical advantages over other existing methods.Recently, Khemakhem et al. ( 2019) introduced a theory of identifiability for deep generative models, based upon which they proposed an identifiable variant of VAEs called iVAE, to learn the distribu-

show abstract

Cubic-Spline Flows

Cited by 20 publications

References 10 publications

Out-of-Distribution Detection of Melanoma using Normalizing Flows

Out-of-Distribution Detection of Melanoma using Normalizing Flows

The frontier of simulation-based inference

Identifying through Flows for Recovering Latent Representations

Contact Info

Product

Resources

About