Statistical inference for high-dimension, low-sample-size data

Abstract

In this paper, we consider statistical inference for high-dimension, low-sample-size (HDLSS) data. We first show that HDLSS data have distinct geometric representations depending on whether or not the data meets a certain boundary condition. We clarify the limit of the conventional principal component analysis (PCA) for HDLSS data. In order to overcome the curse of dimensionality, we introduce two effective PCAs called the noisereduction methodology and the cross-data-matrix (CDM) methodology. We further introduce the extended CDM methodology, which offers an unbiased estimator having small asymptotic variance and low computational cost, for feature parameters appearing in high-dimensional data analysis. We give correlation tests and several inferences on multiclass mean vectors for HDLSS data, and discuss sample size determination to ensure prespecified high accuracy for inference. Finally, we introduce two effective discriminant procedures: the geometric classifier and the distance-based classifier, that can hold misclassification rates less than a threshold. This article originally appeared in Japanese in Sūgaku 65 (3) (2013), 225-247.

High-dimensional Statistical Analysis and Its Application to an ALMA Map of NGC 253

“…However, at the end of the last millennium, a strong need for an analysis method for such data arose (e.g., Golub et al 1999). Then, during the subsequent decade, a new statistical method to tackle HDLSS problems was significantly developed, and is known as the highdimensional statistical analysis (e.g., Aoshima & Yata 2017;). This paper is organized twofold.…”

“…Then, in their sequence of papers, M. Aoshima, K. Yata, A. Ishii and collaborators further developed a new framework of statistical methodology to tackle this type of problem (e.g., Aoshima & Yata 2011;Yata & Aoshima 2012Aoshima & Yata 2014Ishii et al 2016;Aoshima & Yata 2019, and references therein). Readers who are interested in the mathematical background of high-dimensional statistical analysis itself are guided to some reviews, (e.g., Aoshima & Yata 2017;.…”