Sinkhorn Barycenters with Free Support via Frank-Wolfe Algorithm

Luise, Giulia; Salzo, Saverio; Pontil, Massimiliano; Ciliberto, Carlo

doi:10.48550/arxiv.1905.13194

Cited by 4 publications

(4 citation statements)

References 27 publications

(49 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This yields a meta-framework which allows to employ fixed-support Wasserstein barycenter algorithms for fixed-support (p, C)-barycenter computation. One can also modify more general free support methods [Cuturi and Doucet, 2014, Ge et al, 2019, Luise et al, 2019, which usually alternate between updating the support set of the barycenter and its weights on this set, to provide approximate (p, C)-barycenters. However, the necessary position updates usually explicitly or implicitly rely on the fact that the barycentric application T J,p can be computed efficiently.…”

Section: Iterative Algorithms and The Multi-scale Approachmentioning

confidence: 99%

Kantorovich-Rubinstein distance and barycenter for finitely supported measures: Foundations and Algorithms

Heinemann¹,

Klatt²,

Munk³

2021

Preprint

View full text Add to dashboard Cite

The purpose of this paper is to provide a systematic discussion of a generalized barycenter based on a variant of unbalanced optimal transport (UOT) that defines a distance between general non-negative measures by allowing for mass creation and destruction modeled by some cost parameter. We refer to them as Kantorovich-Rubinstein (KR) barycenter and distance. We restrict our analysis to finite ground spaces as demanded for any KR based real world data analysis. In particular, we detail the influence of the cost parameter to structural properties of the KR barycenter and the KR distance. For the latter we highlight a closed form solution on ultrametric trees. The support of KR barycenters of finitely supported measures turns out to be finite and its geometry to be explicitly specified by the support of the input data measures. Additionally, we prove the existence of sparse KR barycenters and discuss potential computational approaches. The performance of the KR barycenter is compared to the OT barycenter on a multitude of synthetic data sets. We also consider barycenters based on the recently introduced Gaussian Hellinger-Kantorovich and Wasserstein-Fisher-Rao distances.

show abstract

Section: Iterative Algorithms and The Multi-scale Approachmentioning

confidence: 99%

Kantorovich-Rubinstein distance and barycenter for finitely supported measures: Foundations and Algorithms

Heinemann¹,

Klatt²,

Munk³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Its exact computation requires the solution of a multimarginal OT problems [Carlier, 2003, Gangbo andSwiech, 1998], it has a polynomial complexity [Altschuler and Boix-Adsera, 2021] but does not scale to large input distributions. In low dimension, one can discretize the barycenter support and use standard solvers such as Frank-Wolfe methods [Luise et al, 2019], entropic regularization Doucet, 2014, Janati et al, 2020], interior point methods [Ge et al, 2019] and stochastic gradient descent [Li et al, 2015]. These approaches could be generalized to compute UOT barycenters.…”

Section: Introductionmentioning

confidence: 99%

Faster Unbalanced Optimal Transport: Translation invariant Sinkhorn and 1-D Frank-Wolfe

Séjourné¹,

Vialard²,

Peyré³

2022

Preprint

View full text Add to dashboard Cite

Unbalanced optimal transport (UOT) extends optimal transport (OT) to take into account mass variations to compare distributions. This is crucial to make OT successful in ML applications, making it robust to data normalization and outliers. The baseline algorithm is Sinkhorn, but its convergence speed might be significantly slower for UOT than for OT. In this work, we identify the cause for this deficiency, namely the lack of a global normalization of the iterates, which equivalently corresponds to a translation of the dual OT potentials. Our first contribution leverages this idea to develop a provably accelerated Sinkhorn algorithm (coined "translation invariant Sinkhorn") for UOT, bridging the computational gap with OT. Our second contribution focusses on 1-D UOT and proposes a Frank-Wolfe solver applied to this translation invariant formulation. The linear oracle of each steps amounts to solving a 1-D OT problems, resulting in a linear time complexity per iteration. Our last contribution extends this method to the computation of UOT barycenter of 1-D measures. Numerical simulations showcase the convergence speed improvement brought by these three approaches.

show abstract

“…to choose the steepest descent direction in the admissible set. This approach was studied in Luise et al (2019) within the context of regularized optimal transport, where the regularized Wasserstein barycenter (also known as the Sinkhorn barycenter) minimizes a sum of Sinkhorn divergences. When the admissible set of directions forms a Reproducing Kernel Hilbert Space (RKHS) with suitable kernel, Shen et al (2020) propose to generate samples distributed according to the Sinkhorn barycenter by iterative pushforward of an initial measure with the map id R d − h • d, where h is a step size and d is the direction of steepest descent in the RKHS.…”

Section: Introductionmentioning

confidence: 99%

Sampling From the Wasserstein Barycenter

Daaloul,

Gouic,

Liandrat

et al. 2021

Preprint

View full text Add to dashboard Cite

This work presents an algorithm to sample from the Wasserstein barycenter of absolutely continuous measures. Our method is based on the gradient flow of the multimarginal formulation of the Wasserstein barycenter, with an additive penalization to account for the marginal constraints. We prove that the minimum of this penalized multimarginal formulation is achieved for a coupling that is close to the Wasserstein barycenter. The performances of the algorithm are showcased in several settings.

show abstract

Sinkhorn Barycenters with Free Support via Frank-Wolfe Algorithm

Cited by 4 publications

References 27 publications

Kantorovich-Rubinstein distance and barycenter for finitely supported measures: Foundations and Algorithms

Kantorovich-Rubinstein distance and barycenter for finitely supported measures: Foundations and Algorithms

Faster Unbalanced Optimal Transport: Translation invariant Sinkhorn and 1-D Frank-Wolfe

Sampling From the Wasserstein Barycenter

Contact Info

Product

Resources

About