An Explicit Mean-Covariance Parameterization for Multivariate Response Linear Regression

Abstract

We develop a new method to fit the multivariate response linear regression model that exploits a parametric link between the regression coefficient matrix and the error covariance matrix. Specifically, we assume that the correlations between entries in the multivariate error random vector are proportional to the cosines of the angles between their corresponding regression coefficient matrix columns, so as the angle between two regression coefficient matrix columns decreases, the correlation between the corresponding errors increases. We highlight two models under which this parameterization arises: a latent variable reduced-rank regression model and the errors-in-variables regression model. We propose a novel non-convex weighted residual sum of squares criterion which exploits this parameterization and admits a new class of penalized estimators. The optimization is solved with an accelerated proximal gradient descent algorithm. Our method is used to study the association between microRNA expression and cancer drug activity measured on the NCI-60 cell lines. An R package implementing our method, MCMVR, is available online.

MLE of Jointly Constrained Mean-Covariance of Multivariate Normal Distributions

“…This factor model interpretation can also be expanded and viewed as the error-in-variable model in the context of multivariate regression (Molstad et al, 2020) where the error covariance matrix and the regression coefficient matrix are parametrically connected. Our model is also a special case of the envelop models in Cook and Zhang (2015).…”

“…As a potential relaxation of the first constraint which is the source of most complications, and for the sake of demonstration, an intermediate constraint Σb = µ for some possibly known vector b is also considered, hoping that it will shed more light on the nature of the constraints. We interpret the constraints in the context of factor and error-in-variable models in multivariate regression (Molstad et al, 2020) where the error covariance matrix and the regression coefficient matrix are parameterically connected.…”