<title>Theoretical and complexity issues for feature set evaluation using boundary methods</title>

Abstract

Boundary Methods (BMs) are a collection of tools used for distribution analysis. This paper explores the theoretical and complexity issues associated with using BMs for Feature Set Evaluation (FSE). First we show the theoretical relationship between Overlap Sum (OS), the BM measure of class separability, and Bayes error (e). This relationship demonstrates the utility of using BMs for FSE. Next, we investigate complexity issues associated with using BMs for FSE and compare with other techniques used for FSE.

Multiresolution estimates of classification complexity

“…Bayes error-based parametric and nonparametric approaches [15] (use of probability distance measure bounds [7], [45], entropy measures [4], [47], nonparametric estimation including k nearest neighbor [6], [10], [25], [46], and Parzen estimation [28], interclass distance measures [9], [13], [41], [47], probability distances such as Bhattacharya [2], Chernoff [5], etc., for multiclass problems [1], [14]). See [13], [43] for a criticism of such approaches; 2. scatter matrices [7], [13]; 3. information-theory-based approaches [26], [44]; 4. boundary methods [30], [31], [32], [35], [36]; 5. correlation-based approaches [33]; 6. nonparametric methods [17], [20]; and 7. feature space partitioning methods [24]. The above approaches to estimating class separability are very different to each other in terms of their methodology, assumptions, and computational complexities.…”