A Stable Alternative to Sinkhorn’s Algorithm for Regularized Optimal Transport

Dvurechensky, Pavel; Gasnikov, Alexander; Omelchenko, Sergey; Tiurin, Alexander

doi:10.1007/978-3-030-49988-4_28

Cited by 18 publications

(18 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Triangles [175,26] and is obtained via the change of the Dual Averaging step (see Section 3.4) to the Bregman Proximal Gradient step. In our presentation of the accelerated method, we consider a particular choice of the the control sequences, i.e., numerical sequences α k , A k from [203,202]. A more general way of constructing such sequences can be found in [35], see also the constants used in the S-CG method described at the end of Section 5.…”

Section: Accelerated Gradient Methodsmentioning

confidence: 99%

“…One can also use the strong convexity assumption to obtain faster convergence rate of the U-A-BPGM either by restarts [180,145], or by incorporating the strong convexity parameter in the steps [57]. The same backtracking line-search can be applied in a much simpler way if one knows that f is L f -smooth with some unknown Lipschitz constant or to achieve acceleration in practice caused by a pessimistic estimate for L f [54,196,200,202,214,215,203]. The idea is to use standard exact "descent Lemma" inequality in each step of the accelerated method.…”

Section: Universal Accelerated Methodsmentioning

confidence: 99%

“…Similarly to (6.18) the objective in the Lagrange dual problem has Lipschitz gradient, yet the challenge is that the feasible set in the dual problem is not bounded. Despite that it is possible to obtain accelerated primal-dual methods [195,196,199,200,201,202,203,204,155,205,132,206]. In particular, this allows to obtain improved complexity bounds for different types of optimal transport problems [202,207,205,140,208,209,210,211,212].…”

Section: Smooth Minimization Of Non-smooth Functionsmentioning

confidence: 99%

See 2 more Smart Citations

First-Order Methods for Convex Optimization

Dvurechensky¹,

Staudigl²,

Shtern³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

First-order methods for solving convex optimization problems have been at the forefront of mathematical optimization in the last 20 years. The rapid development of this important class of algorithms is motivated by the success stories reported in various applications, including most importantly machine learning, signal processing, imaging and control theory. First-order methods have the potential to provide low accuracy solutions at low computational complexity which makes them an attractive set of tools in large-scale optimization problems. In this survey we cover a number of key developments in gradient-based optimization methods. This includes non-Euclidean extensions of the classical proximal gradient method, and its accelerated versions. Additionally we survey recent developments within the class of projection-free methods, and proximal versions of primal-dual schemes. We give complete proofs for various key results, and highlight the unifying aspects of several optimization algorithms.

show abstract

Section: Accelerated Gradient Methodsmentioning

confidence: 99%

Section: Universal Accelerated Methodsmentioning

confidence: 99%

Section: Smooth Minimization Of Non-smooth Functionsmentioning

confidence: 99%

See 1 more Smart Citation

First-Order Methods for Convex Optimization

Dvurechensky¹,

Staudigl²,

Shtern³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Empirical studies showed, with the help of the Sinkhorn algorithm, the regularized OT problem can be solved reliably and efficiently in the cases when

n \approx 10^{4}

(Flamary & Courty, 2017; Genevay et al, 2016). Recently, many studies are developed upon the Sinkhorn algorithm for faster calculations (Altschuler et al, 2017, 2019; Dvurechensky et al, 2018; Lin et al, 2019). For example, Altschuler et al (2019) proposed the Nys‐sink algorithm, which combined the Sinkhorn algorithm with the Nyström method, a popular technique for low‐rank matrix decomposition (Gittens & Mahoney, 2016; Musco & Musco, 2017; S. Wang & Zhang, 2013; Williams & Seeger, 2001).…”

Section: Problem Formulationmentioning

confidence: 99%

Projection‐based techniques for high‐dimensional optimal transport problems

Zhang

Zhong

et al. 2022

WIREs Computational Stats

View full text Add to dashboard Cite

Optimal transport (OT) methods seek a transformation map (or plan) between two probability measures, such that the transformation has the minimum transportation cost. Such a minimum transport cost, with a certain power transform, is called the Wasserstein distance. Recently, OT methods have drawn great attention in statistics, machine learning, and computer science, especially in deep generative neural networks. Despite its broad applications, the estimation of high-dimensional Wasserstein distances is a well-known challenging problem owing to the curse-of-dimensionality. There are some cutting-edge projection-based techniques that tackle high-dimensional OT problems. Three major approaches of such techniques are introduced, respectively, the slicing approach, the iterative projection approach, and the projection robust OT approach. Open challenges are discussed at the end of the review.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Dimension Reduction Statistical Learning and Exploratory Methods of the Data Sciences > Manifold Learning

show abstract

“…Then we analyze the convergence rate of our primal-dual version of the Algorithm 1. Note that the primal-dual analysis of the existing accelerated methods [66,2,7,12,23,24,25,26,33,50] does not apply since the dual problem is a stochastic optimization problem and we use additional randomization. Algorithm 6.1 of [18] applied to the dual problem (22) with stochastic inexact oracle ∇ Φ(λ, ξ, ξ) is listed as Algorithm C3.…”

Section: Proof Of Theoremmentioning

confidence: 99%

Distributed Optimization with Quantization for Computing Wasserstein Barycenters

Krawtschenko¹,

Uribe²,

Gasnikov³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

We study the problem of the decentralized computation of entropy-regularized semidiscrete Wasserstein barycenters over a network. Building upon recent primal-dual approaches, we propose a sampling gradient quantization scheme that allows efficient communication and computation of approximate barycenters where the factor distributions are stored distributedly on arbitrary networks. The communication and algorithmic complexity of the proposed algorithm are shown, with explicit dependency on the size of the support, the number of distributions, and the desired accuracy. Numerical results validate our algorithmic analysis.

show abstract

A Stable Alternative to Sinkhorn’s Algorithm for Regularized Optimal Transport

Cited by 18 publications

References 30 publications

First-Order Methods for Convex Optimization

First-Order Methods for Convex Optimization

Projection‐based techniques for high‐dimensional optimal transport problems

Distributed Optimization with Quantization for Computing Wasserstein Barycenters

Contact Info

Product

Resources

About