The generalized equivalence of regularization and min–max robustification in linear mixed models

Abstract

The connection between regularization and min–max robustification in the presence of unobservable covariate measurement errors in linear mixed models is addressed. We prove that regularized model parameter estimation is equivalent to robust loss minimization under a min–max approach. On the example of the LASSO, Ridge regression, and the Elastic Net, we derive uncertainty sets that characterize the feasible noise that can be added to a given estimation problem. These sets allow us to determine measurement error bounds without distribution assumptions. A conservative Jackknife estimator of the mean squared error in this setting is proposed. We further derive conditions under which min-max robust estimation of model parameters is consistent. The theoretical findings are supported by a Monte Carlo simulation study under multiple measurement error scenarios.

l2-Penalized temporal logit-mixed models for the estimation of regional obesity prevalence over time

“…On that note, Burgard et al. 63 showed that l 2 -penalization is equivalent to a robustification against arbitrary measurement errors. However, their results apply to linear-mixed models, and it is uncertain whether they can be extended to GLMMs.…”

Robust prediction of domain compositions from uncertain data using isometric logratio transformations in a penalized multivariate Fay–Herriot model

“…In particular, we state a framework called min-max robust regression that arises from RO and allows for robust model parameter estimates in the presence of unknown auxiliary data contamination. The following descriptions draw from Bertsimas and Copenhaver (2018) and Burgard et al (2020).…”

“…For further details and applications of RO, see El Ghaoui and Lebret (1997), Markovsky and Van Huffel (2007), and Ben-Tal, El Ghaoui, and Nemirovski (2009). Recent theoretical developments showed that RO has a natural connection to penalized regression (Bertsimas & Copenhaver, 2018;Burgard, Krause, Kreber, & Morales, 2020;Krause, 2019). More precisely, RO can be performed by using norm-based penalization in linear models, such as the elastic net proposed by Zou and Hastie (2005).…”

“…More precisely, RO can be performed by using norm-based penalization in linear models, such as the elastic net proposed by Zou and Hastie (2005). Burgard et al (2020) show that a wide range of regularized regressions provide robust predictions against errors in the covariates. Hereby, the errors in the covariates do not have to follow a known distribution as in Burgard et al (2020aBurgard et al ( , 2020b.…”

“…In these cases, again, the aforementioned methods cannot be applied. By contrast, the robust optimization approach by Burgard et al (2020) is particularly suited for this standard situation in small area estimation.…”