The regularized tau estimator: A robust and efficient solution to ill-posed linear inverse problems

Abstract

Linear inverse problems are ubiquitous. Often the measurements do not follow a Gaussian distribution. Additionally, a model matrix with a large condition number can complicate the problem further by making it ill-posed. In this case, the performance of popular estimators may deteriorate significantly. We have developed a new estimator that is both nearly optimal in the presence of Gaussian errors while being also robust against outliers. Furthermore, it obtains meaningful estimates when the problem is ill-posed through the inclusion of ℓ1 and ℓ2 regularizations. The computation of our estimate involves minimizing a non-convex objective function. Hence, we are not guaranteed to find the global minimum in a reasonable amount of time. Thus, we propose two algorithms that converge to a good local minimum in a reasonable (and adjustable) amount of time, as an approximation of the global minimum. We also analyze how the introduction of the regularization term affects the statistical properties of our estimator. We confirm high robustness against outliers and asymptotic efficiency for Gaussian distributions by deriving measures of robustness such as the influence function, sensitivity curve, bias, asymptotic variance, and mean square error. We verify the theoretical results using numerical experiments and show that the proposed estimator outperforms recently proposed methods, especially for increasing amounts of outlier contamination. Python code for all of the algorithms are available online in the spirit of reproducible research.

Robust, Sparse and Scalable Inference Using Bootstrap and Variable Selection Fusion

“…Modern science and technology involve datasets that either have high-dimensional or are subject to undesirable large perturbations such as heavy-tailed errors, outliers or highleverage points. The τ -Lasso estimator [33], [34] is a regularized robust estimator whose objective function comprises a regularization term to deal with high-dimensional models and a robust empirical loss to deal with outliers and high-leverage points. While 1 -norm regularization of the τ -Lasso sets some coefficients to zero, which is a desired property, it also severely shrinks the estimated coefficients associated with larger true coefficients.…”

“…In particular, we characterize the theoretical properties of both the τ -Lasso estimator and the adaptive τ -Lasso estimator. Martinez-Camara et al [33], [34] originally proposed the τ -Lasso estimator and derived its influence function. We also shed light on the asymptotic properties of τ -Lasso and show that adaptive τ -Lasso enjoys certain robustness and oracle properties.…”