Bilinear and Trilinear Regression Models with Structured Covariance Matrices

Abstract

Joseph Nzabanita (2015). Bilinear and Trilinear Regression Models with Structured Covariance Matrices Doctoral dissertation. This thesis focuses on the problem of estimating parameters in bilinear and trilinear regression models in which random errors are normally distributed. In these models the covariance matrix has a Kronecker product structure and some factor matrices may be linearly structured. Most of techniques in statistical modeling rely on the assumption that data were generated from the normal distribution. Whereas real data may not be exactly normal, the normal distributions serve as a useful approximation to the true distribution. The modeling of normally distributed data relies heavily on the estimation of the mean and the covariance matrix. The interest of considering various structures for the covariance matrices in different statistical models is partly driven by the idea that altering the covariance structure of a parametric model alters the variances of the model's estimated mean parameters.Firstly, we consider the extended growth curve model with a linearly structured covariance matrix. In general there is no problem to estimate the covariance matrix when it is completely unknown. However, problems arise when one has to take into account that there exists a structure generated by a few number of parameters. An estimation procedure that handles linear structured covariance matrices is proposed. The idea is first to estimate the covariance matrix when it may be used to define an inner product in a regression space and thereafter re-estimate it when it should be interpreted as a dispersion matrix. This idea is exploited by decomposing the residual space, the orthogonal complement to the design space, into orthogonal subspaces. Studying residuals obtained from projections of observations on these subspaces yields explicit consistent estimators of the covariance matrix. An explicit consistent estimator of the mean is also proposed.Secondly, we study a bilinear regression model with matrix normally distributed random errors. For those models, the dispersion matrix follows a Kronecker product structure and it can be used, for example, to model data with spatio-temporal relationships. The aim is to estimate the parameters of the model when, in addition, one covariance matrix is assumed to be linearly structured. On the basis of n independent observations from a matrix normal distribution, estimating equations, a flip-flop relation, are established.At last, the models based on normally distributed random third order tensors are studied. These models are useful in analyzing 3-dimensional data arrays. The 3-dimensional data arrays may be obtained when, for example, a single response is sampled in a 3-D space or in a 2-D space and time, multiple responses are recorded in a 2-D space or in a 1-D space and time. In some studies the analysis is done using the tensor normal model, where the focus is on the estimation of the variance-covariance matrix which has a Kronecker structure. Little attention...

Optimal sample complexity for stable matrix recovery

“…where Ω X is a cone (1) - (6). However, stability on a bounded constraint set only requires that Ω B has a bound on its covering number, N Ω B (ρ) ≤ C 1 ( 1 ρ ) d1 , where d 1 is an upper bound on the number of degrees of freedom.…”

“…Next, we prove Proposition 3.2. We split the proof into six parts, bounding the covering numbers of different Ω B 's corresponding to different Ω X 's defined by (1) - (6).…”

“…In matrix completion [4], blind deconvolution via lifting [5], and bilinear regression [6], the measure matrices have rank-1. In phase retrieval via lifting [7], and in covariance matrix estimation via sketching [8], the measurement matrices are symmetric (or Hermitian) rank-1 matrices.…”