Approximate Bayesian Computation with the Wasserstein Distance

Abstract

Summary A growing number of generative statistical models do not permit the numerical evaluation of their likelihood functions. Approximate Bayesian computation has become a popular approach to overcome this issue, in which one simulates synthetic data sets given parameters and compares summaries of these data sets with the corresponding observed values. We propose to avoid the use of summaries and the ensuing loss of information by instead using the Wasserstein distance between the empirical distributions of the observed and synthetic data. This generalizes the well‐known approach of using order statistics within approximate Bayesian computation to arbitrary dimensions. We describe how recently developed approximations of the Wasserstein distance allow the method to scale to realistic data sizes, and we propose a new distance based on the Hilbert space filling curve. We provide a theoretical study of the method proposed, describing consistency as the threshold goes to 0 while the observations are kept fixed, and concentration properties as the number of observations grows. Various extensions to time series data are discussed. The approach is illustrated on various examples, including univariate and multivariate g‐and‐k distributions, a toggle switch model from systems biology, a queuing model and a Lévy‐driven stochastic volatility model.

Model Misspecification in Approximate Bayesian Computation: Consequences and Diagnostics

“…Therefore, theorem 1 is an extension of theorem 1 in Frazier et al (2018) to the case of misspecified models. In addition, we note that theorem 1 above is similar to theorem 4.3 in Bernton et al (2019) for ABC inference based on the Wasserstein distance. The validity of each of these results requires that the map θ → b.θ/ is injective.…”

“…In addition, we note that theorem 1 above is similar to theorem 4.3 in Bernton et al . () for ABC inference based on the Wasserstein distance. The validity of each of these results requires that the map θ ↦ b ( θ ) is injective.…”

Approximate Bayesian Computation with the Sliced-Wasserstein Distance

“…Wasserstein-ABC [9] is a variant of ABC (1) that uses W p , p ≥ 1 between the empirical distributions of the observed and synthetic data, in place of the discrepancy measure D between summaries. To make this method scalable to any dataset size, [9] introduces a new approximation of (2), the Hilbert distance, which extends the idea behind the computation of W p in 1D to higher dimensions, by sorting samples according to their projection obtained via the Hilbert space-filling curve. This alternative can be computed in O(n log(n)), but yields accurate approximations only for low dimensions [9].…”

“…To make this method scalable to any dataset size, [9] introduces a new approximation of (2), the Hilbert distance, which extends the idea behind the computation of W p in 1D to higher dimensions, by sorting samples according to their projection obtained via the Hilbert space-filling curve. This alternative can be computed in O(n log(n)), but yields accurate approximations only for low dimensions [9]. They also use a second approximation, the swapping distance, based on an iterative greedy swapping algorithm.…”

“…data from µ σ for 100 values of σ 2 equispaced between 0.1 and 9. We then compute W 2 and SW 2 between the empirical distributions of the samples, and the swapping and Hilbert approximations presented in [9], for d ∈ {2, 10, 100} and 1000 observations. W 2 between two Gaussian measures has an analytical formula, which boils down in our setting to: W 2 2 (µ σ , µ σ ) = d(σ − σ) 2 , and we approximate the exact SW using a Monte Carlo approximation of:…”