Gradient descent algorithms for Bures-Wasserstein barycenters

Chewi, Sinho; Maunu, Tyler; Rigollet, Philippe; Stromme, Austin J.

doi:10.48550/arxiv.2001.01700

Cited by 5 publications

(6 citation statements)

References 22 publications

(36 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, it is possible to obtain guarantees in a restricted setting by establishing a Polyak-Lojasiewicz type inequality. In particular, assuming all µ i are Gaussian distributions with positive-definite covariance matrices, it is shown that the gradient-descent algorithm admits a linear convergence rate [Chewi et al, 2020].…”

Section: Methods and Algorithmsmentioning

confidence: 99%

Scalable Computations of Wasserstein Barycenter via Input Convex Neural Networks

Fan¹,

Taghvaei²,

Chen

2020

Preprint

View full text Add to dashboard Cite

Wasserstein Barycenter is a principled approach to represent the weighted mean of a given set of probability distributions, utilizing the geometry induced by optimal transport. In this work, we present a novel scalable algorithm to approximate the Wasserstein Barycenters aiming at high-dimensional applications in machine learning. Our proposed algorithm is based on the Kantorovich dual formulation of the 2-Wasserstein distance as well as a recent neural network architecture, input convex neural network, that is known to parametrize convex functions. The distinguishing features of our method are: i) it only requires samples from the marginal distributions; ii) unlike the existing semi-discrete approaches, it represents the Barycenter with a generative model; iii) it allows to compute the barycenter with arbitrary weights after one training session. We demonstrate the efficacy of our algorithm by comparing it with the state-of-art methods in multiple experiments.Preprint. Under review.

show abstract

Section: Methods and Algorithmsmentioning

confidence: 99%

Scalable Computations of Wasserstein Barycenter via Input Convex Neural Networks

Fan¹,

Taghvaei²,

Chen

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Second, while this paper and much of the literature focuses on computing Wasserstein barycenters of discrete distributions, there is also an interesting line of work on computing barycenters of continous distributions. In order to ensure that µ i and ν have concise representations for computational purposes, the distributions are typically restricted to Gaussians, in which case specialized algorithms can be designed; see e.g., [5,12].…”

Section: Previous Workmentioning

confidence: 99%

Wasserstein barycenters are NP-hard to compute

Altschuler,

Boix-Adsera

2021

Preprint

View full text Add to dashboard Cite

The problem of computing Wasserstein barycenters (a.k.a. Optimal Transport barycenters) has attracted considerable recent attention due to many applications in data science. While there exist polynomial-time algorithms in any fixed dimension, all known runtimes suffer exponentially in the dimension. It is an open question whether this exponential dependence is improvable to a polynomial dependence. This paper proves that unless P = NP, the answer is no. This uncovers a "curse of dimensionality" for Wasserstein barycenter computation which does not occur for Optimal Transport computation. Moreover, our hardness results for computing Wasserstein barycenters extend to approximate computation, to seemingly simple cases of the problem, and to averaging probability distributions in other Optimal Transport metrics.

show abstract

“…This implies that the barycenter coincides with such data distribution leading to a significantly simpler problem. In the case where all measures are Gaussians, Chewi et al [12] derive explicitely the gradients of the Wasserstein barycenter functional with respect to the mean and variance of the barycenter and use SGD to learn it.…”

Section: Inductive Biasesmentioning

confidence: 99%

Estimating Barycenters of Measures in High Dimensions

Cohen,

Arbel,

Deisenroth

2020

Preprint

View full text Add to dashboard Cite

Barycentric averaging is a principled way of summarizing populations of measures. Existing algorithms for estimating barycenters typically parametrize them as weighted sums of Diracs and optimize their weights and/or locations. However, these approaches do not scale to high-dimensional settings due to the curse of dimensionality. In this paper, we propose a scalable and general algorithm for estimating barycenters of measures in high dimensions. The key idea is to turn the optimization over measures into an optimization over generative models, introducing inductive biases that allow the method to scale while still accurately estimating barycenters. We prove local convergence under mild assumptions on the discrepancy showing that the approach is well-posed. We demonstrate that our method is fast, achieves good performance on low-dimensional problems, and scales to high-dimensional settings. In particular, our approach is the first to be used to estimate barycenters in thousands of dimensions.Preprint. Under review.

show abstract

Gradient descent algorithms for Bures-Wasserstein barycenters

Cited by 5 publications

References 22 publications

Scalable Computations of Wasserstein Barycenter via Input Convex Neural Networks

Scalable Computations of Wasserstein Barycenter via Input Convex Neural Networks

Wasserstein barycenters are NP-hard to compute

Estimating Barycenters of Measures in High Dimensions

Contact Info

Product

Resources

About