Sparse envelope model: efficient estimation and response variable selection in multivariate linear regression

Su, Zhihua; Zhu, Guangyu; Chen, X.; Yang, Yi

doi:10.1093/biomet/asw036

Cited by 53 publications

(39 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, for estimation we modify the objective functions in equations (4.4) and (4.5) to

f_{L | 2} (G) = log false| G^{normalT} {boldS}_{res | 2} boldG false| + log false| G^{normalT} {boldS}_{Y | 2}^{- 1} boldG false| + {italicλ}_{1} \sum_{i = 1}^{r} w_{1, i} false| false| G_{i} {| |}_{2},

f_{R | 1} (U) = log false| U^{normalT} {boldS}_{res | 1} boldU false| + log false| U^{normalT} {boldS}_{Y | 1}^{- 1} boldU false| + {italicλ}_{2} \sum_{j = 1}^{m} w_{2, j} false| false| U_{j} {| |}_{2},

where G i is the i th row of G and U j is the j th row of U . We attain the minimization of

f_{L | 2} false(boldG false)

and

f_{R | 1} false(boldU false)

by a non‐Grassman and blockwise co‐ordinate descent algorithm (Su et al ., ; Cook et al ., ) to update G and U iteratively until convergence. Once L and R have been estimated, the rest of the parameters are estimated in the same way as in the non‐sparse setting.…”

Section: Sparse Matrix Variate Regressionmentioning

confidence: 99%

“…In particular, Su et al . () studied sparse multivariate regression with an envelope structure. However, no investigation has been done in the matrix variate setting.…”

Section: Sparse Matrix Variate Regressionmentioning

confidence: 99%

“…Sparse regression models have been widely studied in multivariate settings (e.g. Turlach et al (2005), Rothman et al (2010), Chen and Huang (2012) and Su et al (2016)). In particular, Su et al (2016) studied sparse multivariate regression with an envelope structure.…”

Section: Sparse Matrix Variate Regressionmentioning

confidence: 99%

See 2 more Smart Citations

Matrix Variate Regressions and Envelope Models

Ding

Cook

2017

Journal of the Royal Statistical Society Series B: Statistical Methodology

View full text Add to dashboard Cite

Summary Modern technology often generates data with complex structures in which both response and explanatory variables are matrix valued. Existing methods in the literature can tackle matrix‐valued predictors but are rather limited for matrix‐valued responses. We study matrix variate regressions for such data, where the response Y on each experimental unit is a random matrix and the predictor X can be either a scalar, a vector or a matrix, treated as non‐stochastic in terms of the conditional distribution Y|X. We propose models for matrix variate regressions and then develop envelope extensions of these models. Under the envelope framework, redundant variation can be eliminated in estimation and the number of parameters can be notably reduced when the matrix variate dimension is large, possibly resulting in significant gains in efficiency. The methods proposed are applicable to high dimensional settings.

show abstract

“…Therefore, for estimation we modify the objective functions in equations (4.4) and (4.5) to

f_{L | 2} (G) = log false| G^{normalT} {boldS}_{res | 2} boldG false| + log false| G^{normalT} {boldS}_{Y | 2}^{- 1} boldG false| + {italicλ}_{1} \sum_{i = 1}^{r} w_{1, i} false| false| G_{i} {| |}_{2},

f_{R | 1} (U) = log false| U^{normalT} {boldS}_{res | 1} boldU false| + log false| U^{normalT} {boldS}_{Y | 1}^{- 1} boldU false| + {italicλ}_{2} \sum_{j = 1}^{m} w_{2, j} false| false| U_{j} {| |}_{2},

where G i is the i th row of G and U j is the j th row of U . We attain the minimization of

f_{L | 2} false(boldG false)

and

f_{R | 1} false(boldU false)

Section: Sparse Matrix Variate Regressionmentioning

confidence: 99%

“…In particular, Su et al . () studied sparse multivariate regression with an envelope structure. However, no investigation has been done in the matrix variate setting.…”

Section: Sparse Matrix Variate Regressionmentioning

confidence: 99%

See 1 more Smart Citation

Matrix Variate Regressions and Envelope Models

Ding

Cook

2017

Journal of the Royal Statistical Society Series B: Statistical Methodology

View full text Add to dashboard Cite

show abstract

“…Other forms of singular value penalization were considered in, e.g., Mukherjee and Zhu (2011), Chen et al (2013), and Zhou and Li (2014). In addition, some recent efforts further improve low-rank methods by incorporating error covariance modeling, such as envelope models (Cook et al, 2015), or by utilizing variable selection (Bunea et al, 2012;Chen and Huang, 2012;Su et al, 2016).…”

Section: =1mentioning

confidence: 99%

Integrative Multi-View Regression: Bridging Group-Sparse and Low-Rank Models

Liu

Chen

2018

Biometrics

View full text Add to dashboard Cite

Multi‐view data have been routinely collected in various fields of science and engineering. A general problem is to study the predictive association between multivariate responses and multi‐view predictor sets, all of which can be of high dimensionality. It is likely that only a few views are relevant to prediction, and the predictors within each relevant view contribute to the prediction collectively rather than sparsely. We cast this new problem under the familiar multivariate regression framework and propose an integrative reduced‐rank regression (iRRR), where each view has its own low‐rank coefficient matrix. As such, latent features are extracted from each view in a supervised fashion. For model estimation, we develop a convex composite nuclear norm penalization approach, which admits an efficient algorithm via alternating direction method of multipliers. Extensions to non‐Gaussian and incomplete data are discussed. Theoretically, we derive non‐asymptotic oracle bounds of iRRR under a restricted eigenvalue condition. Our results recover oracle bounds of several special cases of iRRR including Lasso, group Lasso, and nuclear norm penalized regression. Therefore, iRRR seamlessly bridges group‐sparse and low‐rank methods and can achieve substantially faster convergence rate under realistic settings of multi‐view learning. Simulation studies and an application in the Longitudinal Studies of Aging further showcase the efficacy of the proposed methods.

show abstract

“…Cook et al () incorporated the idea of enveloping into reduced‐rank regression and demonstrated efficiency gains. Su et al () developed a sparse envelope model that performs response variable selection under the envelope model. Khare et al () proposed a comprehensive Bayesian framework for estimation and model selection in envelope models.…”

Section: Introductionmentioning

confidence: 99%

Vector autoregression and envelope model

Wang

Ding

2018

Stat

View full text Add to dashboard Cite

Vector autoregression is an important technique for modelling multivariate time series and has been widely used in a variety of applications. Owing to its fast growth of parameters with the dimension of the time series vector, dimension reduction is often desirable in multivariate time series analysis. The envelope model is a new approach to achieve dimension reduction and allows efficient estimation in multivariate analysis. In this paper, we provide the first work to explore the application and extension of envelope models to multivariate time series data. We present the envelope and partial envelope formulations for vector autoregression and elaborate model selection, parameter estimation and asymptotic results. Simulations and real data analysis demonstrate the efficiency gains of the envelope vector autoregression models compared with the standard models in terms of estimation. Meanwhile, the envelope models can excel in prediction improvement.

show abstract

Sparse envelope model: efficient estimation and response variable selection in multivariate linear regression

Cited by 53 publications

References 28 publications

Matrix Variate Regressions and Envelope Models

Matrix Variate Regressions and Envelope Models

Integrative Multi-View Regression: Bridging Group-Sparse and Low-Rank Models

Vector autoregression and envelope model

Contact Info

Product

Resources

About