Simpler Grassmannian optimization

Abstract

There are two widely used models for the Grassmannian Gr(k, n), as the set of equivalence classes of orthogonal matrices O(n)/ O(k) × O(n − k) , and as the set of trace-k projection matrices {P ∈ R n×n : P T = P = P 2 , tr(P ) = k}. The former, standard in manifold optimization, has the advantage of giving numerically stable algorithms but the disadvantage of having to work with equivalence classes of matrices. The latter, widely used in coding theory and probability, has the advantage of using actual matrices (as opposed to equivalence classes) but working with projection matrices is numerically unstable. We present an alternative that has both advantages and suffers from neither of the disadvantages; by representing k-dimensional subspaces as symmetric orthogonal matrices of trace 2k − n, we obtainAs with the other two models, we show that differential geometric objects and operations -tangent vector, metric, normal vector, exponential map, geodesic, parallel transport, gradient, Hessian, etc -have closed-form analytic expressions that are computable with standard numerical linear algebra. In the proposed model, these expressions are considerably simpler, a result of representing Gr(k, n) as a linear section of a compact matrix Lie group O(n), and can be computed with at most one qr decomposition and one exponential of a special skew-symmetric matrix that takes only O nk(n − k) time. In particular, we completely avoid eigen-and singular value decompositions in our steepest descent, conjugate gradient, quasi-Newton, and Newton methods for the Grassmannian.

A Grassmann manifold handbook: basic geometry and computational aspects

“…In the next proposition, we complete the picture by providing a formula for the parallel transport on the Grassmannian from the projector perspective. Note that this formula is similar to the parallel transport formula in the preprint [39]. Applying the horizontal lift to the parallel transport equation leads to the formula also found in [21].…”

“…Yet, the research literature on the basis/ONB perspective and the projector perspective is rather disjoint. The recent preprint [39] proposes yet another perspective, namely representing p-dimensional subspaces as symmetric orthogonal matrices of trace 2 p − n. This approach corresponds to a scaling and translation of the projector matrices in the vector space of symmetric matrices, hence it yields very similar formulae.…”

A Grassmann Manifold Handbook: Basic Geometry and Computational Aspects

“…In the next proposition, we complete the picture by providing a formula for the parallel transport on the Grassmannian from the projector perspective. Note that this formula is similar to the parallel transport formula in the preprint [34].…”

“…Yet, the research literature on the basis/ONB perspective and the projector perspective is rather disjoint. The recent preprint [34] proposes yet another perspective, namely representing p-dimensional subspaces as symmetric orthogonal matrices of trace 2p − n. This approach corresponds to a scaling and translation of the projector matrices in the vector space of symmetric matrices, hence it yields very similar formulae.…”

The Grassmannian of affine subspaces

“…As a result, in areas connected to applications such as coding theory [7,10], machine learning [12], optimization [39,42], and statistics [8], the Grassmannian is typically modelled as a set of projection matrices: Gr(k, R n ) ∼ = {P ∈ S 2 (R n ) : P 2 = P, tr(P ) = k}; (1) or, more recently, as a set of involution matrices [25]:…”