Distributed Optimization with Quantization for Computing Wasserstein Barycenters

Abstract

We study the problem of the decentralized computation of entropy-regularized semidiscrete Wasserstein barycenters over a network. Building upon recent primal-dual approaches, we propose a sampling gradient quantization scheme that allows efficient communication and computation of approximate barycenters where the factor distributions are stored distributedly on arbitrary networks. The communication and algorithmic complexity of the proposed algorithm are shown, with explicit dependency on the size of the support, the number of distributions, and the desired accuracy. Numerical results validate our algorithmic analysis.

Decentralized Convex Optimization on Time-Varying Networks with Application to Wasserstein Barycenters

“…Starting with [8,9] it was observed that decentralized methods with dual oracle are well suited for the WB problem. In the cycle of subsequent papers [9][10][11][12][13] different decentralized accelerated (randomized) algorithms were proposed for dual WB problem. In [20] the authors propose to reformulate the WB problem as a bilinear saddle-point problem (SPP).…”

“…If communication network changes over time, then the affine-consensus constraints also change over time, and so does the dual problem. This essentially requires to solve a family of dual problems, which is not possible by the accelerated gradient methods or the Mirror-Prox algorithm as in [9][10][11][12][13][14].…”

“…The computation of the Wasserstein distance is already a large-scale optimization problem, and to calculate the WB, one introduces a second optimization level as the WB minimizes the sum of Wasserstein distances to a set of probability measures. At this point, distributed optimization algorithms turned out to be efficient to scale-up the computations of the WB when the data is distributedly stored by a computational network [7][8][9][10][11][12][13][14]. On the other hand, the nature of the data-generating process may be distributed itself.…”

First-Order Methods for Convex Optimization

“…Despite that it is possible to obtain accelerated primal-dual methods [195,196,199,200,201,202,203,204,155,205,132,206]. In particular, this allows to obtain improved complexity bounds for different types of optimal transport problems [202,207,205,140,208,209,210,211,212].…”

Numerical methods for model-free pricing in finance, optimal transport, and cyber risk management

“…. , 𝜇 𝑁 are unknown with sample access, and develop stochastic optimization algorithms for approximating a Wasserstein barycenter with fixed support; see, e.g., [Claici, Chien, and Solomon, 2018, Staib, Claici, Solomon, and Jegelka, 2017, Krawtschenko, Uribe, Gasnikov, and Dvurechensky, 2020, Zhang, Qian, and Xie, 2023. Recently, numerical methods for continuous Wasserstein barycenter based on neural network parametrization or generative neural networks have been developed; see, e.g., [Cohen et al, 2020a, Fan, Taghvaei, and Chen, 2020, Korotin, Egiazarian, Li, and Burnaev, 2022, Korotin, Li, Solomon, and Burnaev, 2021, Li, Genevay, Yurochkin, and Solomon, 2020.…”