Positive definite matrices and the S-divergence

Abstract

Abstract. Positive definite matrices abound in a dazzling variety of applications. This ubiquity can be in part attributed to their rich geometric structure: positive definite matrices form a selfdual convex cone whose strict interior is a Riemannian manifold. The manifold view is endowed with a "natural" distance function while the conic view is not. Nevertheless, drawing motivation from the conic view, we introduce the S-Divergence as a "natural" distance-like function on the open cone of positive definite matrices. We motivate the S-divergence via a sequence of results that connect it to the Riemannian distance. In particular, we show that (a) this divergence is the square of a distance; and (b) that it has several geometric properties similar to those of the Riemannian distance, though without being computationally as demanding. The S-Divergence is even more intriguing: although nonconvex, we can still compute matrix means and medians using it to global optimality. We complement our results with some numerical experiments illustrating our theorems and our optimization algorithm for computing matrix medians.Key words. Bregman matrix divergence; Log Determinant; Stein Divergence; Jensen-Bregman divergence; matrix geometric mean; matrix median; nonpositive curvature 1. Introduction. Hermitian positive definite (HPD) matrices are a noncommutative generalization of positive reals. They abound in a multitude of applications and exhibit attractive geometric properties-e.g., they form a differentiable Riemannian (also Finslerian) manifold [10,33] that is a well-studied example of a manifold of nonpositive curvature [17, Ch.10]. HPD matrices possess even more structure: (i) they embody a canonical higher-rank symmetric space [51]; and (ii) their closure forms a closed, self-dual convex cone.The convex conic view enjoys great importance in convex optimization [6,43,44] and in nonlinear Perron-Frobenius theory [40]; symmetric spaces are important in algebra, analysis [32,39,51], and optimization [43,52]; while the manifold view (Riemannian or Finslerian) plays diverse roles-see [10, Ch.6] and [46].The manifold view is equipped with a with a "natural" distance function while the conic view is not. Nevertheless, drawing motivation from the convex conic view, we introduce the S-Divergence as a "natural" distance-like function on the open cone of positive definite matrices. Indeed, we prove a sequence of results connecting the S-Divergence to the Riemannian distance. Most importantly, we show that (a) this divergence is the square of a distance; and (b) that it has several geometric properties in common with the Riemannian distance, without being numerically as demanding. This builds an informal link between the manifold and conic views of HPD matrices.1.1. Background and notation. We begin by fixing notation. The letter H denotes some Hilbert space, usually just C n . The inner product between two vectors x and y in H is x, y := x * y (x * denotes 'conjugate transpose'). The set of n × n Hermitian matrices is denoted as H...

Transformations on positive definite matrices preserving generalized distance measures

“…(As a matter of curiosity we mention that in [3] it was conjectured that δ S not a metric, shortly after that in [2] the opposite was claimed, and finally, Sra has shown that δ S is indeed a true metric on P n .) In [15] he has pointed out the importance of this new distance function. Among others, he has emphasized that δ S is a useful substitute of the widely applied geodesic distance δ R , it respects a non-Euclidean geometry of a rather similar kind, but, compared to the case of δ R , the calculation of δ S is easier, it is much less time and capacity demanding which is a really considerable advantage from the computational points view.…”

Riemannian Sparse Coding for Positive Definite Matrices

“…We will compare against six other methods, namely (i) log-Euclidean sparse coding (LE-SC) [18] that projects the data into the Log-Euclidean symmetric space, followed by sparse coding the matrices as Euclidean objects, (ii) Frob-SC, in which the manifold structure is discarded, (iii) Stein-Kernel-SC [20] using a kernel defined by the symmetric Stein divergence [21], (iv) Log-Euclidean Kernel-SC which is similar to (iii) but uses the log-Euclidean kernel [23], (v) tensor sparse coding (TSC) [15] which uses the log-determinant divergence, and generalized dictionary learning (GDL) [16].…”

“…In [20], a kernelized sparse coding scheme is presented for SPD matrices using the Stein divergence [21] for generating the underlying kernel function. But this divergence does not induce a kernel for all bandwidths.…”

Image Analysis on Symmetric Positive Definite Manifolds

“…We can also use the recently introduced Stein divergence [Sra12] to determine simi-larities between points on the SPD manifold. Its symmetrised form is:…”

“…Very recently, Sra et al introduced the Stein kernel using Bregman matrix divergence as follows [Sra12]:…”