Random matrix theory in statistics: A review

Paul, Debashis; Aue, Alexander

doi:10.1016/j.jspi.2013.09.005

Cited by 163 publications

(122 citation statements)

References 257 publications

Supporting

Mentioning

115

Contrasting

Order By: Relevance

“…Finally, the distribution of eigenvalues uncovered by the extreme high-dimension G 50 850 and G 50 8750 completed matrices displayed spectral distributions consistent with the behavior of spiked covariance models of high-dimensional covariance matrices (Paul and Aue 2014). The term spiked refers to a small number of dimensions that have large eigenvalues, while the vast majority of dimensions have eigenvalues equal in value to some arbitrary small number.…”

Section: The Distribution Of Genetic Variance In High Dimensionssupporting

confidence: 52%

“…Disciplines as diverse as ecology, mathematical physics, signal processing, and finance struggle with the problem of estimating large covariance matrices (Paul and Aue 2014), although in most cases they tend to be sample covariance matrices rather than the more derived variance component matrices that represent G. The use of covariance matrices to describe the distribution of variance in high dimensions rests on the additional assumption of multivariate normality (MVN), which also underlies much of quantitative genetic theory and the multivariate response to selection (Lande 1979). Substantial deviation from the MVN assumption can potentially obscure the distribution of variance across trait combinations (see fig.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Phenome-Wide Distribution of Genetic Variance

Blows¹,

Allen²,

Chenoweth³

et al. 2015

The American Naturalist

View full text Add to dashboard Cite

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org. abstract: A general observation emerging from estimates of additive genetic variance in sets of functionally or developmentally related traits is that much of the genetic variance is restricted to few trait combinations as a consequence of genetic covariance among traits. While this biased distribution of genetic variance among functionally related traits is now well documented, how it translates to the broader phenome and therefore any trait combination under selection in a given environment is unknown. We show that 8,750 gene expression traits measured in adult male Drosophila serrata exhibit widespread genetic covariance among random sets of five traits, implying that pleiotropy is common. Ultimately, to understand the phenome-wide distribution of genetic variance, very large additive genetic variancecovariance matrices (G) are required to be estimated. We draw upon recent advances in matrix theory for completing high-dimensional matrices to estimate the 8,750-trait G and show that large numbers of gene expression traits genetically covary as a consequence of a single genetic factor. Using gene ontology term enrichment analysis, we show that the major axis of genetic variance among expression traits successfully identified genetic covariance among genes involved in multiple modes of transcriptional regulation. Our approach provides a practical empirical framework for the genetic analysis of highdimensional phenome-wide trait sets and for the investigation of the extent of high-dimensional genetic constraint.

show abstract

Section: The Distribution Of Genetic Variance In High Dimensionssupporting

confidence: 52%

Section: Introductionmentioning

confidence: 99%

The Phenome-Wide Distribution of Genetic Variance

Blows¹,

Allen²,

Chenoweth³

et al. 2015

The American Naturalist

View full text Add to dashboard Cite

show abstract

“…This effect is generally obtained in high-dimensional data and becomes more severe when c becomes smaller. [17][18][19][20] When c is not much larger than 1, the sample estimate is numerically ill conditioned, ie, inverting it to estimate the precision matrix will amplify estimation error. Additionally, when c < 1, matrix S loses full rank.…”

Section: The Sample Covariance Estimator Is a Poor Estimator For Himentioning

confidence: 99%

“…Several algorithms have been proposed to solve the problem Equation 19. To date, the most popular approach is the graphical lasso (glasso), where a solution to (Equation 19) is found by solving a series of coupled regression problems in an iterative fashion.…”

Section: The Graphical Lassomentioning

confidence: 99%

An overview of large‐dimensional covariance and precision matrix estimators with applications in chemometrics

Engel

Buydens

Blanchet

2017

Journal of Chemometrics

View full text Add to dashboard Cite

The covariance matrix (or its inverse, the precision matrix) is central to many chemometric techniques. Traditional sample estimators perform poorly for high-dimensional data such as metabolomics data. Because of this, many traditional inference techniques break down or produce unreliable results. In this paper, we selectively review several modern estimators of the covariance and precision matrix that improve upon the traditional sample estimator. We focus on 3 general techniques: eigenvalue-shrinkage estimation, ridge-type estimation, and structured estimation. These methods rely on different assumptions regarding the structure of the covariance or precision matrix. Various examples, in particular using metabolomics data, are used to compare these techniques and to demonstrate that in concert with, eg, principal component analysis, multivariate analysis of variance, and Gaussian graphical models, better results are obtained. KEYWORDSeigenvalue shrinkage, metabolomics, ridge-type estimation, sparse covariance matrix, sparse precision matrix

show abstract

“…The conventional ITC-like methods have been developed for the large-scale sensor array in the framework of random matrix theory, which considers the asymptotic condition , → ∞ with / → ∈ (0, ∞) and provides more accurate descriptions for the sample eigenvalues of the high dimensional observations [6]. B. Nadler modified the Akaike information criterion (AIC) via increasing the penalty term based on the probability distribution of the largest sample eigenvalue [7].…”

Section: Introductionmentioning

confidence: 99%

Source enumeration in large arrays using corrected Rao's score test and relatively few samples

Liu

Sun

Liu

2017

2017 25th European Signal Processing Conference (EUSIPCO)

View full text Add to dashboard Cite

Abstract-We focus on the problem of source enumeration in large arrays with relatively few samples, which is solved in this paper by using a statistic of corrected Rao's score test (CRST) via the generalized Bayesian information criterion (GBIC). Under the white noise assumption, the covariance matrix of the noise subspace components of the observations is proportional to an identity matrix, and this structure can be tested by the CRST statistic for the sphericity hypothesis test. The observations are decomposed into signal and noise subspace components by unitary coordinate transformation under a presumptive number of sources. Only when there is no signal in the presumptive noise subspace components, the corresponding CRST statistic is asymptotic normal distribution. The CRST statistic of the presumptive noise subspace components also is a statistic of the sample eigenvalues, and can be used as the statistic in the GBIC for estimating the number of sources. Simulation results demonstrate that the proposed method can achieve more accurate detection of the number of sources in the case of a large number of sensors with relatively few samples, especially when the number of samples is smaller than the number of sensors.

show abstract

Random matrix theory in statistics: A review

Cited by 163 publications

References 257 publications

The Phenome-Wide Distribution of Genetic Variance

The Phenome-Wide Distribution of Genetic Variance

An overview of large‐dimensional covariance and precision matrix estimators with applications in chemometrics

Source enumeration in large arrays using corrected Rao's score test and relatively few samples

Contact Info

Product

Resources

About