Reduced-rank regression for the multivariate linear model

Izenman, Alan Julian

doi:10.1016/0047-259x(75)90042-1

Cited by 482 publications

(285 citation statements)

References 9 publications

Supporting

Mentioning

275

Contrasting

Unclassified

Order By: Relevance

“…Other interesting approaches which may be pursued in the context of multi-task learning include multivariate linear models in statistics such as reduced rank regression (Izenman 1975), partial least squares (Wold et al 1984) and canonical correlation analysis (Hotelling 1936) (see also Breiman and Friedman 1997). These methods are based on generalized eigenvalue problems-see, for example, (Borga 1998, Chap.…”

Section: Discussionmentioning

confidence: 99%

Convex multi-task feature learning

2008

View full text Add to dashboard Cite

We present a method for learning sparse representations shared across multiple tasks. This method is a generalization of the well-known single-task 1-norm regularization. It is based on a novel non-convex regularizer which controls the number of learned features common across the tasks. We prove that the method is equivalent to solving a convex optimization problem for which there is an iterative algorithm which converges to an optimal solution. The algorithm has a simple interpretation: it alternately performs a supervised and an unsupervised step, where in the former step it learns task-specific functions and in the latter step it learns common-across-tasks sparse representations for these functions. We also provide an extension of the algorithm which learns sparse nonlinear representations using kernels. We report experiments on simulated and real data sets which demonstrate that the proposed method can both improve the performance relative to learning each task independently and lead to a few learned features common across related tasks. Our algorithm can also be used, as a special case, to simply select-not learn-a few common variables across the tasks.

show abstract

Section: Discussionmentioning

confidence: 99%

Convex multi-task feature learning

2008

View full text Add to dashboard Cite

show abstract

“…For this, we compare the stacked multi-output model to CCA and reduced rank regression (RRR) to predict clinical scores. CCA 310 finds linear combinations to optimize correlations between X and Y. RRR is a multi-output linear regression that imposes a low-rank constraint on the coefficient matrix (Izenman, 1975). This model seeks common latent factors across the different outputs to improve regression accuracy.…”

Section: (Iii) Stacking Compared To Other Source Combination Strategiesmentioning

confidence: 99%

“…Reduced rank regression 120 (Vounou et al, 2010;Izenman, 1975) is a related multi-output linear-regression. Unlike CCA, it does not discard explained variance during model fitting.…”

Section: Related Workmentioning

confidence: 99%

Joint prediction of multiple scores captures better individual traits from brain images

et al. 2017

View full text Add to dashboard Cite

To probe individual variations in brain organization, population imaging relates features of brain images to rich descriptions of the subjects such as genetic information or behavioral and clinical assessments. Capturing common trends across these measurements is important: they jointly characterize the disease status of patient groups. In particular, mapping imaging features to behavioral scores with predictive models opens the way toward more precise diagnosis. Here we propose to jointly predict all the dimensions (behavioral scores) that make up the individual profiles, using so-called multi-output models. This approach often boosts prediction accuracy by capturing latent shared information across scores. We demonstrate the efficiency of multi-output models on two independent resting-state fMRI datasets targeting different brain disorders (Alzheimer's Disease and schizophrenia). Furthermore, the model with joint prediction generalizes much better to a new cohort: a model learned on one study is more accurately transferred to an independent one. Finally, we show how multi-output models can easily be extended to multi-modal settings, combining heterogeneous data sources for a better overall accuracy.

show abstract

“…RRR is closely related to the analysis of canonical covariates and canonical correlations of Hostelling (1935Hostelling ( , 1936 and Anderson (1951Anderson ( , 1984. Discussion of these relations can be found in Anderson (1951), Izenman (1975), Tso (1981), Davies and Tso (1982), and Reinsel and Velu (1998). Reduced rank models with stationary variables have been analyzed by Velu, Reinsel, and Wichern (1986) and Velu and Reinsel (1987), Gudmundsson (1977) applied reduced rank regression to select linear combinations that best described the average variation in a vector of dependent variables.…”

Section: Introductionmentioning

confidence: 99%

Generalized Reduced Rank Regression

Hansen

2002

SSRN Journal

View full text Add to dashboard Cite

Reduced-rank regression for the multivariate linear model

Cited by 482 publications

References 9 publications

Convex multi-task feature learning

Convex multi-task feature learning

Joint prediction of multiple scores captures better individual traits from brain images

Generalized Reduced Rank Regression

Contact Info

Product

Resources

About