The statistics and mathematics of high dimension low sample size asymptotics

Shen, Dan; Shen, Haipeng; Zhu, Hongtu; Marron, J. S.

doi:10.5705/ss.202015.0088

Cited by 42 publications

(58 citation statements)

References 38 publications

(66 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The proof, which is given in in Appendix D, is based on Weyl's inequality and a useful variant of the Davis-Kahan theorem . We notice that some preceding works (Onatski, 2012;Shen et al, 2016;Wang and Fan, 2017) have provided similar results under a weaker pervasiveness assumption which allows p/n → ∞ in any manner and the spiked eigenvalues {λ } K =1 are allowed to grow slower than p so long as c = p/(nλ ) is bounded.…”

Section: U -Type Covariance Estimationmentioning

confidence: 56%

FarmTest: Factor-Adjusted Robust Multiple Testing With Approximate False Discovery Control

Fan

Yuan

Sun

et al. 2019

Journal of the American Statistical Association

View full text Add to dashboard Cite

Large-scale multiple testing with correlated and heavy-tailed data arises in a wide range of research areas from genomics, medical imaging to finance. Conventional methods for estimating the false discovery proportion (FDP) often ignore the effect of heavy-tailedness and the dependence structure among test statistics, and thus may lead to inefficient or even inconsistent estimation. Also, the commonly imposed joint normality assumption is arguably too stringent for many applications. To address these challenges, in this paper we propose a Factor-Adjusted Robust Multiple Testing (FarmTest) procedure for largescale simultaneous inference with control of the false discovery proportion. We demonstrate that robust factor adjustments are extremely important in both controlling the FDP and improving the power. We identify general conditions under which the proposed method produces consistent estimate of the FDP. As a byproduct that is of independent interest, we establish an exponentialtype deviation inequality for a robust U -type covariance estimator under the spectral norm. Extensive numerical experiments demonstrate the advantage of the proposed method over several state-of-the-art methods especially when the data are generated from heavy-tailed distributions. The proposed procedures are implemented in the R-package FarmTest.

show abstract

Section: U -Type Covariance Estimationmentioning

confidence: 56%

FarmTest: Factor-Adjusted Robust Multiple Testing With Approximate False Discovery Control

Fan

Yuan

Sun

et al. 2019

Journal of the American Statistical Association

View full text Add to dashboard Cite

show abstract

“…A more important point, however, is that PCA does not require more individuals than variables when it is calculated with singular value decomposition (SVD) as opposed to Eigenvector decomposition (Shen et al, 2016;Yata and Aoshima, 2010) for implementation (see the prcomp function in R). These statistical procedures in fact produce equivalent results, although SVD is more numerically accurate for some data and so preferable on those grounds.…”

Section: Vaccinesmentioning

confidence: 99%

A Manual for Cultural Analysis

Matthews¹,

Brown²,

Kennedy³

2018

View full text Add to dashboard Cite

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions. The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.

show abstract

“…Microarray technology is a high-throughput platform for analyzing gene expression profiles in tissues and makes it possible to examine the expression of thousands of genes simultaneously. However, in most cases, a single microarray dataset is high-dimensional low-sample size (HDLSS) data, which indeed presents serious statistical challenges [14]. Therefore, the integration of multiple microarray data sets can increase the sample size and may improve the statistical power.…”

Section: Introductionmentioning

confidence: 99%

In silico analysis of hub genes expressed in the renal tubulointerstitium of immunoglobulin A nephropathy

Pan¹,

Zhou²,

Li³

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: IgA nephropathy (IgAN) is an autoimmune disease and common cause of chronic renal failure worldwide, but its etiology has not been fully elucidated. Tubulointerstitial fibrosis/atrophy is the most reliable histological marker associated with poor outcome of IgAN. However, few studies have focused on gene expression in the renal tubulointerstitium of IgAN.Methods: A systemic search of gene expression among the renal tubulointerstitium in Gene Expression Omnibus (GEO) datasets was performed, and eligible datasets were integrated to investigate the significant genes associated with IgAN. Differentially expressed genes (DEGs) and significant gene modules of weighted gene coexpression network analysis (WGCNA) were identified, and enrichment analysis using the Gene Ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG) databases was performed for interpretation of these genes.Results: Three datasets named GSE32591, GSE35489 and GSE37463 were selected for analysis of the relationships between gene expression in the renal tubulointerstitium and IgAN. Ten genes named ALB, FN1, CCL2, DUSP1, MMP7, JUNB, CTGF, TIMP3, NEU1 and HLA-E were identified as hub genes from DEGs and the significant gene module of WGCNA. Most of these genes have been related to the significant relationships of prognosis of IgAN in previous studies, but many interactions between these genes were unclear.Conclusion: Our study provided a cluster of hub genes significantly expressed in renal tubulointerstitial IgAN that may be novel treatment targets of IgAN.

show abstract

The statistics and mathematics of high dimension low sample size asymptotics

Cited by 42 publications

References 38 publications

FarmTest: Factor-Adjusted Robust Multiple Testing With Approximate False Discovery Control

FarmTest: Factor-Adjusted Robust Multiple Testing With Approximate False Discovery Control

A Manual for Cultural Analysis

In silico analysis of hub genes expressed in the renal tubulointerstitium of immunoglobulin A nephropathy

Contact Info

Product

Resources

About