Improving Variational Inference with Inverse Autoregressive Flow

Kingma, Diederik P.; Salimans, Tim; Józefowicz, Rafał; Chen, Xi; Sutskever, Ilya; Welling, Max

doi:10.48550/arxiv.1606.04934

Cited by 60 publications

(103 citation statements)

References 15 publications

Supporting

Mentioning

103

Contrasting

Order By: Relevance

“…This estimation assumes Standard Normal priors for the likelihood of the latent data, as described in appendix A. There is a great deal of ongoing research into methods to improve the likelihood estimate by changing the latent space priors or improving the posterior approximations of the encoder[2,48,82,83].…”

mentioning

confidence: 99%

Challenges for Unsupervised Anomaly Detection in Particle Physics

Fraser,

Homiller,

Mishra

et al. 2021

Preprint

View full text Add to dashboard Cite

Anomaly detection relies on designing a score to determine whether a particular event is uncharacteristic of a given background distribution. One way to define a score is to use autoencoders, which rely on the ability to reconstruct certain types of data (background) but not others (signals). In this paper, we study some challenges associated with variational autoencoders, such as the dependence on hyperparameters and the metric used, in the context of anomalous signal (top and W ) jets in a QCD background. We find that the hyperparameter choices strongly affect the network performance and that the optimal parameters for one signal are non-optimal for another. In exploring the networks, we uncover a connection between the latent space of a variational autoencoder trained using mean-squared-error and the optimal transport distances within the dataset. We then show that optimal transport distances to representative events in the background dataset can be used directly for anomaly detection, with performance comparable to the autoencoders. Whether using autoencoders or optimal transport distances for anomaly detection, we find that the choices that best represent the background are not necessarily best for signal identification. These challenges with unsupervised anomaly detection bolster the case for additional exploration of semi-supervised or alternative approaches.

show abstract

mentioning

confidence: 99%

Challenges for Unsupervised Anomaly Detection in Particle Physics

Fraser,

Homiller,

Mishra

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…To estimate the density of the dataset, it is proposed to use normalizing flows, which are primarily applied to generative modeling [48][49][50][51][52][53] and are increasingly being used for scientific applications [54,55]. Given a dataset, generative modeling attempts to create new data points that were previously unseen but are distributed like the original dataset.…”

Section: Probability Map Estimationmentioning

confidence: 99%

Uniform-in-Phase-Space Data Selection with Iterative Normalizing Flows

Hassanaly¹,

Perry²,

Müeller³

et al. 2021

Preprint

View full text Add to dashboard Cite

Figure 1: Graphical illustration of the data selection method. Starting from a large dataset (top left scatter plot), a reduced uniform-in-phase-space dataset is constructed (right scatter plot). The reduction method relies on the probability map which is iteratively computed (Step 1 and Step 1 correction). An acceptance probability constructed with the probability map is used to accept or reject each data point (Step 2).

show abstract

“…This results in an additional loss component and a specific estimator for the training algorithm called the Stochastic Gradient Variational Bayes (SGVB) estimator. Researchers have incorporated some more sophisticated posteriors q(z|x) to extend variational autoencoder [14], [21], [23].…”

Section: B Approximate Inferencementioning

confidence: 99%

Assessing Deep Neural Networks as Probability Estimators

Kuo¹,

Rilee²,

Yu³

2021

Preprint

View full text Add to dashboard Cite

Deep Neural Networks (DNNs) have performed admirably in classification tasks. However, the characterization of their classification uncertainties, required for certain applications, has been lacking. In this work, we investigate the issue by assessing DNNs' ability to estimate conditional probabilities and propose a framework for systematic uncertainty characterization. Denoting the input sample as x and the category as y, the classification task of assigning a category y to a given input x can be reduced to the task of estimating the conditional probabilities p(y|x), as approximated by the DNN at its last layer using the softmax function. Since softmax yields a vector whose elements all fall in the interval (0, 1) and sum to 1, it suggests a probabilistic interpretation to the DNN's outcome. Using synthetic and real-world datasets, we look into the impact of various factors, e.g., probability density f (x) and intercategorical sparsity, on the precision of DNNs' estimations of p(y|x), and find that the likelihood probability density and the inter-categorical sparsity have greater impacts than the prior probability to DNNs' classification uncertainty.

show abstract

Improving Variational Inference with Inverse Autoregressive Flow

Cited by 60 publications

References 15 publications

Challenges for Unsupervised Anomaly Detection in Particle Physics

Challenges for Unsupervised Anomaly Detection in Particle Physics

Uniform-in-Phase-Space Data Selection with Iterative Normalizing Flows

Assessing Deep Neural Networks as Probability Estimators

Contact Info

Product

Resources

About