Envelopes and Partial Least Squares Regression

Cook, R. Dennis; Helland, Inge S.; Su, Zhihua

doi:10.1111/rssb.12018

Cited by 115 publications

(158 citation statements)

References 29 publications

(53 reference statements)

Supporting

Mentioning

152

Contrasting

Order By: Relevance

“…We therefore refer to the RRR model as fully supervised. The SupSVD model (2) is also connected with the envelope model that was recently proposed by Cook et al [8] and further developed in [32,7,11,10]. The envelope model is a parsimonious model for multivariate regression that is based on the assumption that variation in the response can be divided into two parts: a material part that is related to the predictor, and ) -an immaterial part that is unrelated to the predictor.…”

Section: Connections With Existing Modelsmentioning

confidence: 99%

Supervised singular value decomposition and its asymptotic properties

Yang

Nobel

et al. 2016

Journal of Multivariate Analysis

View full text Add to dashboard Cite

a b s t r a c tA supervised singular value decomposition (SupSVD) model has been developed for supervised dimension reduction where the low rank structure of the data of interest is potentially driven by additional variables measured on the same set of samples. The SupSVD model can make use of the information in the additional variables to accurately extract underlying structures that are more interpretable. The model is general and includes the principal component analysis model and the reduced rank regression model as two extreme cases. The model is formulated in a hierarchical fashion using latent variables, and a modified expectation-maximization algorithm for parameter estimation is developed, which is computationally efficient. The asymptotic properties for the estimated parameters are derived. We use comprehensive simulations and a real data example to illustrate the advantages of the SupSVD model.

show abstract

Section: Connections With Existing Modelsmentioning

confidence: 99%

Supervised singular value decomposition and its asymptotic properties

Yang

Nobel

et al. 2016

Journal of Multivariate Analysis

View full text Add to dashboard Cite

show abstract

“…Essentially a form of targeted dimension reduction, this process can lead to substantial efficiency gains when the immaterial variation is large relative to the material variation. Cook, Helland, and Su (2013) adapted envelopes to the predictors, and showed that the SIMPLS algorithm for PLS regression converges to an envelope in the predictor space. Following Cook, Li, and Chiaromonte (2010), they demonstrated that using a likelihood-based objective function to separate the material and immaterial variation and to provide an estimator of the coefficient matrix produces clear and often substantial estimative and predictive advantages over SIMPLS.…”

Section: Introductionmentioning

confidence: 99%

“…We also link the simultaneous envelope method with PLS and CCA. In Section 4, we introduce a likelihood-based objective function that includes the objective functions used by Cook, Li, and Chiaromonte (2010) and Cook, Helland, and Su (2013) as special cases. Novel algorithms for estimating a simultaneous envelope are also given in this section.…”

Section: Introductionmentioning

confidence: 99%

Simultaneous Envelopes for Multivariate Linear Regression

Cook

Zhang

2015

Technometrics

View full text Add to dashboard Cite

We introduce envelopes for simultaneously reducing the predictors and the responses in multivariate linear regression, so the regression then depends only on estimated linear combinations of X and Y. We use a likelihood-based objective function for estimating envelopes and then propose algorithms for estimation of a simultaneous envelope as well as for basic Grassmann manifold optimization. The asymptotic properties of the resulting estimator are studied under normality and extended to general distributions. We also investigate likelihood ratio tests and information criteria for determining the simultaneous envelope dimensions. Simulation studies and real data examples show substantial gain over the classical methods, like partial least squares, canonical correlation analysis, and reduced-rank regression. This article has supplementary material available online.

show abstract

“…LSR plays an important underlying role in many extensions, which can be frequently founded in the literature, e.g., L2 norm regularized LSR (RLSR) [2], locally regularized LSR [3], entropy regularized LSR [4], weighted LSR [5], partial least square regression (PLSR) [6], orthogonal LSR [7], lasso [8], and some recent works including [9][10][11], and [12]. Among them, RLSR and lasso are very popular.…”

Section: Introductionmentioning

confidence: 99%