Robust low-rank matrix estimation

Abstract

Many results have been proved for various nuclear norm penalized estimators of the uniform sampling matrix completion problem. However, most of these estimators are not robust: in most of the cases the quadratic loss function and its modifications are used. We consider robust nuclear norm penalized estimators using two well-known robust loss functions: the absolute value loss and the Huber loss. Under several conditions on the sparsity of the problem (i.e. the rank of the parameter matrix) and on the regularity of the risk function sharp and non-sharp oracle inequalities for these estimators are shown to hold with high probability. As a consequence, the asymptotic behavior of the estimators is derived. Similar error bounds are obtained under the assumption of weak sparsity, i.e. the case where the matrix is assumed to be only approximately low-rank. In all our results we consider a high-dimensional setting. In this case, this means that we assume n ≤ pq. Finally, various simulations confirm our theoretical results.MSC 2010 subject classifications: Primary 62J05, 62F30; secondary 62H12.

Estimation bounds and sharp oracle inequalities of regularized procedures with Lipschitz loss functions

“…This condition has been extensively studied in Learning theory (cf. [5,66,49,7,64,23]). We can identify mainly two approaches to study this condition: when the class F is convex and the loss function is "strongly convex", then the risk function inherits this property and automatically satisfies the Bernstein condition (cf.…”

Robust statistical learning with Lipschitz and convex loss functions

“…As the Lipschitz property allows to make only weak assumptions on the outputs, these losses have been quite popular in robust statistics [17]. Empirical risk minimizers (ERM) based on Lipschitz losses such as the Huber loss have received recently an important attention [45,15,2].…”

“…This boundedness is not satisfied in linear regression with unbounded design so the results of [2] don't apply to this basic example such as linear regression with a Gaussian design. To bypass this restriction, the global condition is relaxed into a "local" one as in [15,42], see Assumption 4 below.The main constraint in our results on ERM is the assumption on the design. This constraint can be relaxed by considering alternative estimators based on the "median-of-means" (MOM) principle of [37,9,18,1] and the minmax procedure of [3,5].…”

“…This boundedness is not satisfied in linear regression with unbounded design so the results of [2] don't apply to this basic example such as linear regression with a Gaussian design. To bypass this restriction, the global condition is relaxed into a "local" one as in [15,42], see Assumption 4 below.…”

“…Let F Y |X=x denote the c.d.f. of Y given X = x.Since for all x ∈ X , a → g(x, a) is twice differentiable in its second argument (see Lemma 2.1 in[15]), a second Taylor expansion yieldsP L f = E ∂g(X, a) ∂a (f * (X))(f (X) − f * (X)) + 1 2 x∈X (f (x) − f * (x)) 2 ∂ 2 g(x, a) ∂a 2 (z x )dP X (x)…”