Doubly sparse regression incorporating graphical structure among predictors

Abstract

Recent research has demonstrated that information learned from building a graphical model on the predictor set of a regularized linear regression model can be leveraged to improve prediction of a continuous outcome. In this article, we present a new model that encourages sparsity at both the level of the regression coefficients and the level of individual contributions in a decomposed representation. This model provides parameter estimates with a finite sample error bound and exhibits robustness to errors in the input graph structure. Through a simulation study and the analysis of two real data sets, we demonstrate that our model provides a predictive benefit when compared to previously proposed models. Furthermore, it is a highly flexible model that provides a unified framework for the fitting of many commonly used regularized regression models. The Canadian Journal of Statistics 47: 729–747; 2019 © 2019 Statistical Society of Canada

Graphical-model based high dimensional generalized linear models

“…We first show that R(β) is a norm and decomposable with respect to a pair of subspaces (M, M ⊥ ). By Lemma 3.2 in [36], we note that R(β) ≥ 0 with equality only when β = 0. Then for u ∈ R\{0}, we have the positive homogeneity, i.e., R(uβ) = |u|R(β).…”

“…[44] discussed the hierarchical sparse modeling which utilized the group Lasso and the latent overlapping group Lasso penalty. Different from these works, we consider an undirected graph structure among the predictors [46,35,51]. As predictors in a neighborhood are connected, they are simultaneously effective or not effective for predicting the response.…”

“…It is known that while the predictors are correlated with each other in a neighborhood, they may not be all important predictors to the response variable. To overcome this, Stephenson et al [36] proposed a doubly sparse method (DSRIG) which encourages sparsity both within and among the decomposition under linear regression model with fixed design matrix. They also applied the doubly sparse method to the logistic regression [37] using the predictor duplication (PD) algorithm [27].…”