Sparse Regression Incorporating Graphical Structure Among Predictors

Yu, Guan; Liu, Yufeng

doi:10.1080/01621459.2015.1034319

Cited by 31 publications

(77 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In other words, node i only contributes to the estimation of regression coefficients associated with its neighbours and so learning the support of V ( i ) is analogous to learning the structure of the predictor graph. Yu & Liu () exploit this relation in the sparse regression incorporating graphical structure among predictors (SRIG) model.…”

Section: Methods and Motivationmentioning

confidence: 99%

“…The SRIG model of Yu & Liu () assumes that the predictor graph structure is known. As such, the support of V ( i ) is also known and estimation of the SRIG coefficients proceeds by solving

\begin{matrix} alignleft \\ align-1 & align-2 \underset{β, V^{(i)} : i = 1, …, p}{\arg \min} \{\frac{1}{2 n} ‖ Y - X β ‖_{2}^{2} + λ {falsefalse}_{\sum}^{i = 1} τ_{i} ‖ V^{(i)} ‖_{2}\}, \end{matrix}

where

bold-italicβ = \sum_{i = 1}^{p} {boldV}^{false(i false)}

, λ ≥ 0 is a tuning parameter and τ i is a weight for the i th predictor.…”

Section: Methods and Motivationmentioning

confidence: 99%

“…Equivalently, the set of true (direct) predictors can be thought of as the set of nodes associated with non‐zero regression coefficients in the underlying data‐generating mechanism. Yu & Liu () demonstrated that the finite sample bounds derived for SRIG prediction and estimation errors require that if any node

i \in {scriptJ}_{0}

, then

{scriptN}_{i} \subseteq {scriptJ}_{0}

(Assumption A2 of Yu & Liu ()). In other words, all neighbours of true predictors are also true predictors.…”

Section: Methods and Motivationmentioning

confidence: 99%

“…Therefore, SRIG may produce a non‐zero β 3 , which would be equivalent to adding an edge between X 3 and Y in the augmented graph of Figure a. Through simulation, Yu & Liu () found that the performance of SRIG decreased relative to the LASSO as the assumption that for any node

i \in {scriptJ}_{0}

, then

{scriptN}_{i} \subseteq {scriptJ}_{0}

became increasingly challenged. By allowing small

V_{j}^{false(i false)}

associated with an edge ( i , j ) for which

i \in {scriptJ}_{0}

but

j \notin {scriptJ}_{0}

to shrink to zero, the effects of violations to this assumption can be minimized.…”

Section: Methods and Motivationmentioning

confidence: 99%

“…). Recently, Yu & Liu () introduced structural sparsity among the regression coefficients by assuming a graphical structure of the predictors and then exploiting that structure during minimization.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Doubly sparse regression incorporating graphical structure among predictors

2019

View full text Add to dashboard Cite

Recent research has demonstrated that information learned from building a graphical model on the predictor set of a regularized linear regression model can be leveraged to improve prediction of a continuous outcome. In this article, we present a new model that encourages sparsity at both the level of the regression coefficients and the level of individual contributions in a decomposed representation. This model provides parameter estimates with a finite sample error bound and exhibits robustness to errors in the input graph structure. Through a simulation study and the analysis of two real data sets, we demonstrate that our model provides a predictive benefit when compared to previously proposed models. Furthermore, it is a highly flexible model that provides a unified framework for the fitting of many commonly used regularized regression models. The Canadian Journal of Statistics 47: 729–747; 2019 © 2019 Statistical Society of Canada

show abstract

Section: Methods and Motivationmentioning

confidence: 99%

“…The SRIG model of Yu & Liu () assumes that the predictor graph structure is known. As such, the support of V ( i ) is also known and estimation of the SRIG coefficients proceeds by solving

\begin{matrix} alignleft \\ align-1 & align-2 \underset{β, V^{(i)} : i = 1, …, p}{\arg \min} \{\frac{1}{2 n} ‖ Y - X β ‖_{2}^{2} + λ {falsefalse}_{\sum}^{i = 1} τ_{i} ‖ V^{(i)} ‖_{2}\}, \end{matrix}

where

bold-italicβ = \sum_{i = 1}^{p} {boldV}^{false(i false)}

, λ ≥ 0 is a tuning parameter and τ i is a weight for the i th predictor.…”

Section: Methods and Motivationmentioning

confidence: 99%

i \in {scriptJ}_{0}

, then

{scriptN}_{i} \subseteq {scriptJ}_{0}

(Assumption A2 of Yu & Liu ()). In other words, all neighbours of true predictors are also true predictors.…”

Section: Methods and Motivationmentioning

confidence: 99%

i \in {scriptJ}_{0}

, then

{scriptN}_{i} \subseteq {scriptJ}_{0}

became increasingly challenged. By allowing small

V_{j}^{false(i false)}

associated with an edge ( i , j ) for which

i \in {scriptJ}_{0}

but

j \notin {scriptJ}_{0}

to shrink to zero, the effects of violations to this assumption can be minimized.…”

Section: Methods and Motivationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Doubly sparse regression incorporating graphical structure among predictors

2019

View full text Add to dashboard Cite

show abstract

Supervised Learning

Liu

2021

Wiley StatsRef: Statistics Reference Online

View full text Add to dashboard Cite

Supervised learning is an important area in machine learning. In practice, many problems can be solved using supervised learning techniques to deal with the corresponding covariate–response data. One general goal is to find a model that predicts the response from the covariates well. We focus on supervised learning methods that can be formulated into the optimization of “loss + penalty.” In particular, the loss term keeps the fidelity of the resulting model to the data, while the penalty term penalizing the complexity can prevent the fitted model from overfitting. In this article, we review some commonly used supervised learning methods under this framework. We mainly focus on the statistical models and computational algorithms.

show abstract

Recovery of sums of sparse and dense signals by incorporating graphical structure among predictors

Luo

Liu

2021

Can J Statistics

Self Cite

View full text Add to dashboard Cite

With the abundance of high-dimensional data, sparse regularization techniques are very popular in practice because of the built-in sparsity of their corresponding estimators. However, the success of sparse methods heavily relies on the assumption of sparsity in the underlying model. For models where the sparsity assumption fails, the performance of these sparse methods can be unsatisfactory and misleading. In this article, we consider the perturbed linear model, where the signal is given by the sum of sparse and dense signals. We propose a new penalization-based method, called Gava, to tackle this kind of signal by making use of a graphical structure among model predictors. The proposed Gava method covers several existing methods as special cases. Our numerical examples and theoretical studies demonstrate the effectiveness of the proposed Gava for estimation and prediction.

show abstract

Sparse Regression Incorporating Graphical Structure Among Predictors

Cited by 31 publications

References 45 publications

Doubly sparse regression incorporating graphical structure among predictors

Doubly sparse regression incorporating graphical structure among predictors

Supervised Learning

Recovery of sums of sparse and dense signals by incorporating graphical structure among predictors

Contact Info

Product

Resources

About