The two-to-infinity norm and singular subspace geometry with applications to high-dimensional statistics

Abstract

The singular value matrix decomposition plays a ubiquitous role throughout statistics and related fields. Myriad applications including clustering, classification, and dimensionality reduction involve studying and exploiting the geometric structure of singular values and singular vectors.This paper provides a novel collection of technical and theoretical tools for studying the geometry of singular subspaces using the two-to-infinity norm. Motivated by preliminary deterministic Procrustes analysis, we consider a general matrix perturbation setting in which we derive a new Procrustean matrix decomposition. Together with flexible machinery developed for the two-to-infinity norm, this allows us to conduct a refined analysis of the induced perturbation geometry with respect to the underlying singular vectors even in the presence of singular value multiplicity. Our analysis yields singular vector entrywise perturbation bounds for a range of popular matrix noise models, each of which has a meaningful associated statistical inference task. In addition, we demonstrate how the two-to-infinity norm is the preferred norm in certain statistical settings. Specific applications discussed in this paper include covariance estimation, singular subspace recovery, and multiple graph inference.Both our Procrustean matrix decomposition and the technical machinery developed for the two-to-infinity norm may be of independent interest.

Orthogonal Procrustes and norm-dependent optimality

“…Lemma 4.1 (below) represents a first step in the direction of understanding the relationship between equation (1.3) and the geometry of sin Θ distance. A proof can be found in [6].…”

“…We catalog several preliminary facts about the two-to-infinity norm in the form of several propositions. The proofs are straightforward in nature, and we refer to [6] for additional details. Below, in summary:…”

“…Theorem 4.4 (below) relates the quantity U − V W 2→∞ to both the sin Θ distance and to the row structure of V and V ⊥ . We remark that in various (statistical) settings involving (stochastic) matrix perturbations, the term implicitly bounded by s U,V V ⊥ 2→∞ can be analyzed more delicately to yield an improved leading-order term, as pursued in [5,6].…”

“…Overview and motivation. This paper is intended to serve as a baseline, perturbation-free counterpart to [6], providing both complementary theory and numerics. Here and in [6], we advocate for more widespread consideration of the two-to-infinity subordinate vector norm on matrices and demonstrate how…”

“…As such, the results discussed here hold in broad generality. By way of contrast, stronger results can be obtained under additional structural assumptions, particularly in the classical matrix perturbation setting A := A + E. In statistics, for example, the papers [3,5,6] consider such settings where U (derived from some matrix A) is viewed as a (stochastic) perturbation of V (derived from some matrix A), making it possible to considerably improve (probabilistically) upon certain bounds (e.g., Theorem 4.4).…”