A Geometric Approach to Low-Rank Matrix Completion

Abstract

The low-rank matrix completion problem can be succinctly stated as follows: given a subset of the entries of a matrix, find a low-rank matrix consistent with the observations. While several low-complexity algorithms for matrix completion have been proposed so far, it remains an open problem to devise search procedures with provable performance guarantees for a broad class of matrix models. The standard approach to the problem, which involves the minimization of an objective function defined using the Frobenius metric, has inherent difficulties: the objective function is not continuous and the solution set is not closed. To address this problem, we consider an optimization procedure that searches for a column (or row) space that is geometrically consistent with the partial observations. The geometric objective function is continuous everywhere and the solution set is the closure of the solution set of the Frobenius metric. We also preclude the existence of local minimizers, and hence establish strong performance guarantees, for special completion scenarios, which do not require matrix incoherence or large matrix size.

A Broyden Class of Quasi-Newton Methods for Riemannian Optimization

“…Recently considered applications include matrix completion problems [8,21,12,33], truss optimization [26], finite-element discretization of Cosserat rods [27], matrix mean computation [6,4], and independent component analysis [31,30]. Research efforts to develop and analyze optimization methods on manifolds can be traced back to the work of Luenberger [20]; they concern, among others, steepest-descent methods [20], conjugate gradients [32], Newton's method [32,3], and trust-region methods [1,5]; see also [2] for an overview.…”

Fast singular value thresholding without singular value decomposition

“…These algorithms usually use non-convex formulations instead of the convex nuclear norm minimization (2), and they can be fast and have comparable recoverability to those based on nuclear norm minimization in practice; see, e.g., [6,21,42,56,71].…”

Simultaneous Codeword Optimization (SimCO) for Dictionary Update and Learning

“…Here, the notations U m,1 and G m,1 follow from the convention in [62], [63]. Note that each element in U m,1 is a unit-norm vector while each element in G m,1 is a one-dimensional subspace in R m .…”

“…Our results show that a gradient descent method will not converge to any other stationary points than global minimizers. More recently, the rank-one decomposition problem where λ 2 = λ 3 = · · · = λ m = 0 was studied in [63]. Our proof technique is significantly different as the effects of the eigen-spaces corresponding to λ 2 , · · · , λ m need to be considered for the rank-one approximation problem.…”