Regularized Discrete Optimal Transport

Abstract

Abstract. This article introduces a generalization of the discrete optimal transport, with applications to color image manipulations. This new formulation includes a relaxation of the mass conservation constraint and a regularization term. These two features are crucial for image processing tasks, which necessitate to take into account families of multimodal histograms, with large mass variation across modes. The corresponding relaxed and regularized transportation problem is the solution of a convex optimization problem. Depending on the regularization used, this minimization can be solved using standard linear programming methods or first order proximal splitting schemes. The resulting transportation plan can be used as a color transfer map, which is robust to mass variation across images color palettes. Furthermore, the regularization of the transport plan helps to remove colorization artifacts due to noise amplification. We also extend this framework to the computation of barycenters of distributions. The barycenter is the solution of an optimization problem, which is separately convex with respect to the barycenter and the transportation plans, but not jointly convex. A block coordinate descent scheme converges to a stationary point of the energy. We show that the resulting algorithm can be used for color normalization across several images. The relaxed and regularized barycenter defines a common color palette for those images. Applying color transfer toward this average palette performs a color normalization of the input images.

Optimal Patch Assignment for Statistically Constrained Texture Synthesis

“…Such an idea was first proposed in [9] and refined by [24] to overcome the problem of color inconsistency in color transfer. First, let us recall that an optimal assignment problem such as the one of Problem (4) can be recast as a Linear Sum Assignment Problem [4] of the form…”

Implementation of the Midway Image Equalization

“…A fully satisfying extension to color images should involve optimal transportation (see [5,6]). For the sake of simplicity, we propose in this demo to apply the midway equalization independently on each color channel.…”

“…But, for color images from two different scenes (Figure 9), such a simple extension is not appropriate to produce suitable results. For those cases, the correct generalization of the midway equalization consists in computing Wasserstein barycenters (see [5,6]). …”

Harmonic Flow for Histogram Matching

“…They include most of the variational formulations of the color transfer literature (see e.g. [5], [9], [17], [19], [20], [25]). …”