Properties of the singular, inverse and generalized inverse partitioned Wishart distributions

We consider the high-dimensional discriminant analysis problem. For this problem, different methods have been proposed and justified by establishing exact convergence rates for the classification risk, as well as the ℓ2 convergence results to the discriminative rule. However, sharp theoretical analysis for the variable selection performance of these procedures have not been established, even though model interpretation is of fundamental importance in scientific data analysis. This paper bridges the gap by providing sharp sufficient conditions for consistent variable selection using the sparse discriminant analysis (Mai et al., 2012). Through careful analysis, we establish rates of convergence that are significantly faster than the best known results and admit an optimal scaling of the sample size n, dimensionality p, and sparsity level s in the high-dimensional setting. Sufficient conditions are complemented by the necessary information theoretic limits on the variable selection problem in the context of high-dimensional discriminant analysis. Exploiting a numerical equivalence result, our method also establish the optimal results for the ROAD estimator (Fan et al., 2012) and the sparse optimal scaling estimator (Clemmensen et al., 2011). Furthermore, we analyze an exhaustive search procedure, whose performance serves as a benchmark, and show that it is variable selection consistent under weaker conditions. Extensive simulations demonstrating the sharpness of the bounds are also provided.

show abstract

“…Using Theorem 3 of Bodnar and Okhrin (2008), we compute the density of

{\hat{z}}_{a} = {\hat{boldG}}_{12} {\hat{boldG}}_{22}^{- 1}

conditional on μ̂ T and obtain that…”

Section: A1 Proofs Of Results In Sectionmentioning

confidence: 99%

Optimal Feature Selection in High-Dimensional Discriminant Analysis

Kolar

Liu

2015

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…While this received significant theoretical developments and laid to success in numerous applications, this yields covariance matrices that are not a priori positive definite for finite sample sizes [26], whose inverse thus does not correspond to a Precision matrix. 5 Experiments not reported here lead us to observe that such estimates yield poor asset allocation performance and thus to conclude that they are not suited for that particular purpose. Instead, sparsity is enforced by minimizing a cost function, balancing a data fidelity term and a penalty term based on the l 1 -norm of the non diagonal entries of the estimated matrix, thus following the large bulk of ongoing efforts aiming to address sparse matrix estimation issues, cf.…”

Section: A Direct Sample Estimatesmentioning

confidence: 73%

“…We define two such pairs. For consistency with the previous approaches, we consider Σ (5) ,Π (5) whose definition follows that of Σ (4) ,Π (4) with the return on the market portfolio r m replaced by the first SVD factor, extracted from the p × n matrix which stacks the time series of the returns r k . We further defineΣ (6) , obtained fromΣ (4) , using the two first factors of the SVD.…”

Section: A Direct Sample Estimatesmentioning

confidence: 99%

“…The case where S n does not admit a regular inverse is more difficult to handle, as the sample precision matrix P n is usually estimated using the Moore-Penrose generalized inverse, whose statistic follows the generalized inverse Wishart distribution [5]. To the best of our knowledge, the moments of that distribution do not admit known closed-form expressions (except for cross-sectionally uncorrelated returns) [13].…”

Section: A Direct Sample Estimatesmentioning

confidence: 99%

See 1 more Smart Citation

Covariance Versus Precision Matrix Estimation for Efficient Asset Allocation

Senneret¹,

Malevergne

Abry

et al. 2016

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

1 Asset allocation constitutes one of the most crucial and most challenging tasks in financial engineering, which often requires the estimation of large covariance or precision matrices, from short time span multivariate observations, a mandatory yet difficult step. The present contribution reviews and compares a large selection of estimators for covariance and precision matrices, organized into classes of estimation principles (Direct, Factor, Shrinkage, Sparsity). This includes the theoretical derivation of several additional estimators not available in the literature. Rather than assessing estimation performance from synthetic data based on a priori selected models of questionable practical interest, it is chosen here to evaluate practically the quality of these estimators directly from portfolio selection performance, quantified by financial criteria. Portfolio selection is conducted over two datasets of different natures: a 15-year large subset (within Stoxx Europe 600) of 244 European stock returns, and a 50-year benchmark dataset of 90 US equity portfolios. This large scale comparative study addresses issues such as the relative benefits and difficulties of using robust versus direct estimates, of choosing precision or covariance estimates, of quantifying the impacts of constraints.

show abstract

“…(a) For proving part (a) we make use of Theorem 3(e) of Bodnar and Okhrin [4]. It implies that (e i − r i 1) r i 1)).…”

Section: Appendix Distribution Of V Imentioning

confidence: 99%