CoCoLasso for high-dimensional error-in-variables regression

Abstract

Much theoretical and applied work has been devoted to highdimensional regression with clean data. However, we often face corrupted data in many applications where missing data and measurement errors cannot be ignored. Loh and Wainwright (2012) proposed a non-convex modification of the Lasso for doing high-dimensional regression with noisy and missing data. It is generally agreed that the virtues of convexity contribute fundamentally the success and popularity of the Lasso. In light of this, we propose a new method named CoCoLasso that is convex and can handle a general class of corrupted datasets including the cases of additive measurement error and random missing data. We establish the estimation error bounds of CoCoLasso and its asymptotic sign-consistent selection property. We further elucidate how the standard cross validation techniques can be misleading in presence of measurement error and develop a novel corrected cross-validation technique by using the basic idea in CoCoLasso. The corrected cross-validation has its own importance. We demonstrate the superior performance of our method over the non-convex approach by simulation studies. 1. Introduction. High-dimensional regression has wide applications in various fields such as genomics, finance, medical imaging, climate science, sensor network, etc. The current inventory of high-dimensional regression methods includes Lasso [23], SCAD [11], elastic net [30], adaptive lasso [29] and Dantzig selector [7] among others. The articles [12] and [13] provide an overview of these existing methods while the book by [5] discusses their statistical properties in finer details. The canonical high-dimensional linear regression model assumes that the number of available predictors (p) is larger than the sample size (n), although the true number of relevant predictors (s) is much less than n. The model is expressed as y = Xβ * + w where y = (y 1 , . . . , y n ) ′ is the vector of responses, X = ((x ij )) is the n × p matrix of covariates, β * is a p × 1 sparse coefficient vector with only s non-zero entries and w = (w 1 , . . . , w n ) ′ is the noise vector.Much of the existing theoretical and applied work on high-dimensional MSC 2010 subject classifications: Primary 62J07; secondary 62F12

MEBoost: Variable selection in the presence of measurement error

“…To conclude, we note that while we only considered linear and Poisson regression in this paper, MEBoost can easily be applied to other regression models by, for example, using the estimating equations presented by Nakamura or others that correct for measurement error. In contrast, the approaches of Sørensen and Thoresen and Datta and Zou exploit the structure of the linear regression model, and it is not obvious how they could be extended to the broader family of generalized linear models. The robustness and simplicity of MEBoost, along with its strong performance against other methods in the linear model case, suggests that this novel method is a reliable way to deal with variable selection in the presence of measurement error.…”

“…Sørensen and Thoresen introduced a variation of the Lasso that allows for normally distributed and independent and identically distributed (iid) additive covariate measurement error. Datta and Zou proposed the Convex Conditioned Lasso (CoCoLasso), which corrects for both additive and multiplicative measurement errors in the normal case. Both of these methods are applicable to linear models for continuous outcomes but do not easily extend to regression models for other outcome types (eg, binary or count data).…”

“…We applied MEBoost to data from the Box Lunch Study, a clinical trial in nutrition where caloric intake across a number of food categories was based on self‐report and, hence, measured with error. Furthermore, to evaluate MEBoost, we conducted a simulation study to compare our method to the CoCoLasso proposed by Datta and Zou and the “naive” Lasso that ignores measurement error.…”

“…A recent paper by Datta and Zou proposes an alternative approach that they refer to as the CoCoLasso. Consider the following reformulation of the Lasso problem: …”

“…If has a negative eigenvalue, then this Lasso function would be nonconvex and unbounded. To overcome this obstacle, the key to the CoCoLasso is calculating the projection of onto the space of positive‐definite matrices, ie, …”