Geodesic PCA versus Log-PCA of Histograms in the Wasserstein Space

Abstract

This paper is concerned by the statistical analysis of data sets whose elements are random histograms. For the purpose of learning principal modes of variation from such data, we consider the issue of computing the PCA of histograms with respect to the 2-Wasserstein distance between probability measures. To this end, we propose to compare the methods of log-PCA and geodesic PCA in the Wasserstein space as introduced in [BGKL15, SC15]. Geodesic PCA involves solving a non-convex optimization problem. To solve it approximately, we propose a novel forward-backward algorithm. This allows a detailed comparison between log-PCA and geodesic PCA of one-dimensional histograms, which we carry out using various datasets, and stress the benefits and drawbacks of each method. We extend these results for two-dimensional data and compare both methods in that setting.

Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning

“…Note that problem (22) has a continuous objective function as well as a compact feasible set and is therefore solvable. Any optimal solution (µ , Σ , {α j , θ j , Θ j } j ) can in principle be used to construct an extremal distribution Q that attains the supremum in the Gelbrich risk evaluation problem (18). Specifically, for any j ∈ [J] let Q j be any distribution supported on…”

“…Even though the mixture components Q j , j ∈ [J], are guaranteed to exist, however, one can prove that it is NP-hard to construct them. In other words, even though it is easy to solve (22) and even though any solution of (22) gives rise to a solution Q of the Gelbrich risk evaluation problem (18), constructing Q remains hard. While exactly computable in polynomial time, the Gelbrich risk of a piecewise quadratic loss function may only provide a loose upper bound on the worst-case risk under the Wasserstein ambiguity set, which is often the actual quantity of interest.…”

“…The SDP (23) is easily obtained from (21) by setting J = 1 and noting that Y = Q 1 , y = q 1 and y 0 = 0 at optimality. As usual, a discrete extremal distribution Q for the Gelbrich risk evaluation problem (18) can be derived from the dual of the SDP (23). In the following we denote the mean vector and the covariance matrix of Q by µ and Σ , respectively.…”

Nonlinear model reduction on metric spaces. Application to one-dimensional conservative PDEs in Wasserstein spaces

“…We emphasize nonetheless that the tPCA has important optimality properties in terms of its decay of the approximation error as we describe next. This is probably the reason why numerical methods based on tPCA have drawn significant interest in numerous fields like pattern recognition, shape analysis, medical imaging, computer vision [26,56], and, more recently, statistics [35] and machine learning to study families of histograms and probability densities [18]. We show evidence that this method also carries potential for model reduction of transport dominated systems in Section 6.…”

“…The first approach is however related to the so-called tangent PCA, which has drawn significant interest in numerous fields like pattern recognition, shape analysis, medical imaging, computer vision [26,56]. More recently, it has also been used in statistics [35] and machine learning to study families of histograms and probability densities [18]. Our second approach based on barycenters is entirely novel to the best of our knowledge, and it could be used as an alternative to tPCA in other applications apart from model reduction.…”