Sharp non-asymptotic performance bounds for $$\ell _1$$ ℓ 1 and Huber robust regression estimators

Abstract

A quantitative study of the robustness properties of the 1 and the Huber M-estimator on finite samples is presented. The focus is on the linear model involving a fixed design matrix and additive errors restricted to the dependent variables consisting of noise and sparse outliers. We derive sharp error bounds for the 1 estimator in terms of the leverage constants of a design matrix introduced here. A similar analysis is performed for Huber's estimator using an equivalent problem formulation of independent interest. Our analysis considers outliers of arbitrary magnitude, and we recover breakdown point results as particular cases when outliers diverge. The practical implications of the theoretical analysis are discussed on two real datasets.

The Derivative of Influence Function, Location Breakdown Point, Group Leverage and Regression Residuals' Plots

“…Giloni and Padberg (2002) presented a lower bound on total sum of absolute L 1 -residuals using the total sum of squared L 2 -residuals. Flores (2015) studied for a particular regression model the behavior of L 1 -estimates by comparing them with L 2 -estimates, and introduced leverage constants for a design matrix to determine whether leverage cases are good or bad.…”

“…Rousseeuw and van Zomeren (1990),Ellis and Morgenthaler (1992),Bradu (1997),Flores (2015) andGenton and Hall (2016).…”