A Fast Proximal Point Method for Computing Exact Wasserstein Distance

Abstract

Wasserstein distance plays increasingly important roles in machine learning, stochastic programming and image processing. Major efforts have been under way to address its high computational complexity, some leading to approximate or regularized variations such as Sinkhorn distance. However, as we will demonstrate, regularized variations with large regularization parameter will degradate the performance in several important machine learning applications, and small regularization parameter will fail due to numerical stability issues with existing algorithms. We address this challenge by developing an Inexact Proximal point method for exact Optimal Transport problem (IPOT) with the proximal operator approximately evaluated at each iteration using projections to the probability simplex. The algorithm (a) converges to exact Wasserstein distance with theoretical guarantee and robust regularization parameter selection, (b) alleviates numerical stability issue, (c) has similar computational complexity to Sinkhorn, and (d) avoids the shrinking problem when apply to generative models. Furthermore, a new algorithm is proposed based on IPOT to obtain sharper Wasserstein barycenter.

Gromov-Wasserstein Learning for Graph Matching and Node Embedding

“…The work in (Altschuler et al, 2017) proves that solving the regularized optimal transport based on Sinkhorn projections is with linear convergence. The work in (Xie et al, 2018) further proves that the linear convergence holds even just applying one-step Sinkhorn projection in each updating step (i.e., J = 1). Therefore, the updating steps of the proposed method are with linear convergence.…”

“…We solve it iteratively with the help of a proximal point method. Inspired by the method in (Xie et al, 2018), in the n-th inner iteration we update the target optimal transport via…”

“…Additionally, we can apply the inexact proximal point method (Xie et al, 2018;Chen et al, 2018a), running one-step Sinkhorn-Knopp projection in each inner iteration. Therefore, the complexity of learning optimal transport is O(V 2 D + N V 3 ).…”

An Optimal-Transport-Based Reinforcement Learning Approach for Computation Offloading

“…We transform the original offloading problem into solving the optimization problem of the optimal transport scheme. We define C(α, β) as the objective cost function, where α = N i=1 a i δ x i denotes a discrete probability measurement of the task to be offloaded by the end device, and β = N i=1 b i δ x i denotes another discrete probability measure for tasks that has already been processed [16]. According to the definition of Monge-Kantorovich transport problem, the optimization problem is formulated as follows:…”

“…When faced with large-scale offloading, solving how to map multiple tasks from one space to another simultaneously, rather than just thinking about one task, is the sticking point to OT. Problem ( 8) is challenging to solve because constraint C3 is not linear, and Kantorovich Relaxation can relax the original constraint, allowing multiple tasks to be offloaded to multiple servers [14], [16]. Since the convex optimization problems are always paired, the convex maximization Problem (9) corresponding to the convex minimization Problem ( 8) is given as follows:…”