Supervised classification in high-dimensional space: geometrical, statistical, and asymptotical properties of multivariate data

Abstract

As the number of spectral bands of high spectral resolution data increases, the capability to detect more detailed classes should also increase, and the classification accuracy should increase as well. Often the number of labeled samples used for supervised classification techniques is limited, thus limiting the precision with which class characteristics can be estimated. As the number of spectral bands becomes large, the limitation on performance imposed by the limited number of training samples can become severe. A number of techniques for case-specific feature extraction have been developed to reduce dimensionality without loss of class separability. Most of these techniques require the estimation of statistics at full dimensionality in order to extract relevant features for classification. If the number of training samples is not adequately large, the estimation of parameters in high dimensional data will not be accurate enough. As a result, the estimated features may not be as effective as they could be. This suggests the need for reducing the dimensionality via a preprocessing method that takes into consideration high dimensional feature space properties. Such reduction should enable the estimation of feature extraction parameters to be more accurate. Using a technique referred to as Projection Pursuit, such an algorithm has been developed. This technique is able to bypass many of the problems of the limitation of small numbers of training samples by making the computations in a lower dimensional space, and optimizing a function called the projection index. A current limitation on this method is that as the number of dimensions increases, it is highly probable to find a local maximum 1 Work reported herein was funded in part by NASA Grant NAGW-3924.

A projection pursuit algorithm to classify individuals using fMRI data: Application to schizophrenia

“…Efficient algorithms are needed in order to extract the robust spatial and spatiotemporal features in the data identifying regions in the brain that best discriminate the classes. Jimenez and Landgrebe (1998) demonstrated that high dimensional space is mostly empty and pointed out that the useful information can be extracted more easily in a lower-dimensional subspace using Projection Pursuit (PP) algorithms (Jimenez and Landgrebe, 1999). PP is a proposed solution for problems where the classification is not accurate due to the limited number of training samples in a high dimensional space.…”

The Kernel Common Vector Method: A Novel Nonlinear Subspace Classifier for Pattern Recognition

“…The estimation of density functions may not be overly reliable for these cases since the dimensionality of the sample space and the number of samples per class are comparable. It has been reported that the number of samples in each class must be at least ten times the dimensionality of the sample space for a reliable density estimation [30], [31]. Thus, our proposed kernel subspace classifier may be a good choice in such cases.…”

Classification of Large Biomedical Data Using ANNs Based on BFGS Method

“…The problem of dimensionality reduction appears in many fields of artificial intelligence such as data mining, data compression and data visualization, moderating the curse of dimensionality and other undesired properties of high dimensional spaces [14]. Given a dataset X = [x 1 , x 2 , ... , x n ] ∈ R n×D consists of n datavectors x i with dimensionality D and has intrinsic dimension d (with d << D).…”