Adaptive color transfer with relaxed optimal transport

Abstract

This paper studies the problem of color transfer between images using optimal transport techniques. While being a generic framework to handle statistics properly, it is also known to be sensitive to noise and outliers, and is not suitable for direct application to images without additional post-processing regularization to remove artifacts. To tackle these issues, we propose to directly deal with the regularity of the transport map and the spatial consistency of the reconstruction. Our approach is based on the relaxed and regularized discrete optimal transport method of [8]. We extend this work by (i) modeling the spatial distribution of colors within the image domain and (ii) tuning automatically the relaxation parameters. Experiments on real images demonstrate the capacity of our model to adapt itself to the considered data.

Optimal Mass Transport: Signal processing and machine-learning applications

“…The color transfer problem, on the other hand, is concerned with pushing the three-dimensional color distribution of the first image into the second one. This problem can also be formulated as an optimal transport problem as demonstrated in [41] (see [24] for more references. ).…”

“…This is due to the fact that a color transfer map may not transfer the colors of neighboring pixels in a coherent manner, and may introduce artifacts in the color transferred image. Therefore, the color transfer map is often regularized to make the transfer map spatially coherent [41]. Figure 7 shows a simple example of grayvalue and color transfer via optimal transport framework.…”

“…These techniques have been used to simplify and augment the accuracy of numerous pattern recognition-related problems. Some examples covered in this tutorial include image retrieval [32, 44], signal and image representation [25, 27, 40, 50], inverse problems [30], cancer detection [4, 39], texture and color modeling [18, 41], shape and image registration [22, 29], and machine learning [12, 17, 19, 28, 36, 42], to name a few. This tutorial is meant to serve as an introductory guide to those wishing to familiarize themselves with these emerging techniques.…”

Optimal Patch Assignment for Statistically Constrained Texture Synthesis

“…Such an idea was first proposed in [9] and refined by [24] to overcome the problem of color inconsistency in color transfer. First, let us recall that an optimal assignment problem such as the one of Problem (4) can be recast as a Linear Sum Assignment Problem [4] of the form…”

Rotation Matrix Sampling Scheme for Multidimensional Probability Distribution Transfer

“…Similarly, in (Rabin et al, 2014), Rabin et al also used the optimal transport technique to address the problem of statistical property transfer. Especially, the relaxed and regularized discrete optimal transport method was utilized.…”