<title>Comparative study of the Gaussian form of the Bhattacharyya metric for ATR performance evaluation</title>

Abstract

The Gaussian form of the Bhattachaiyya distance measure is being used by some in the automatic target recognition (ATR) community to select features and to estimate an upper performance bound for ATR algorithms.One reason for the popularity of this measure is that it is readily computed. This paper shows through both empirical and analytic results the inadequacy ofthis metric.Empirical results are obtained by processing ADTS field data through both the Gaussian form of the Bhattacharyya distance and a nonparametric error estimation scheme. Analytic results are obtained by deriving the Gaussian form of the Bhattachaiyya distance metric for distributions other than Gaussian. These results show that the Gaussian form ofthe Bhattacharyya distance cannot be trusted to provide a reliable upper performance bound.Additional empirical and analytic results show by using a nonparametric performance estimator that when the data is transformed to be more Gaussian the Bhattacharyya metric gives better performance estimates. The transformations discussed are the power transform and a Mode Seeker that decomposes the data into Gaussian modes.A conclusion is that tools can be and should be developed that improve the utility of the Bhattacharyya metric mainly since they provide useful information about the distribution of the data. The major conclusion is that even with these tools nonparametric error estimation techniques are superior. The nonparametric performance bounds are more reliable, and the proper use of the Bhattacharyya metric depends upon considerable knowledge of the data distributions.

ATR Performance Modeling and Estimation

“…A comparative study specifically for ATR performance evaluation [12] found that nonparametric error estimation techniques are superior to those which use the Gaussian form of the Bhattacharyya metric to select features and to estimate upper bounds on ATR performance. To demonstrate this conclusion, Williams used data from the Advanced Detection Technology Sensor (ADTS, 33 GHz, 1-foot resolution) to analyze the distinguishability of grassy and treed areas in SAR imagery.…”

Multiresolution estimates of classification complexity

“…The approaches include 1. Bayes error-based parametric and nonparametric approaches [15] (use of probability distance measure bounds [7], [45], entropy measures [4], [47], nonparametric estimation including k nearest neighbor [6], [10], [25], [46], and Parzen estimation [28], interclass distance measures [9], [13], [41], [47], probability distances such as Bhattacharya [2], Chernoff [5], etc., for multiclass problems [1], [14]). See [13], [43] for a criticism of such approaches; 2. scatter matrices [7], [13]; 3. information-theory-based approaches [26], [44]; 4. boundary methods [30], [31], [32], [35], [36]; 5. correlation-based approaches [33]; 6. nonparametric methods [17], [20]; and 7. feature space partitioning methods [24].…”