Comparing Classifier's Performance Based on Confidence Interval of the ROC

Abstract

This paper proposes a new methodology for comparing two performance methods based on confidence interval for the ROC curve. The methods performed and compared are two algorithms for face recognition. The novelty of the paper is threefold: i) designing a methodology for the comparison of decision making algorithms via confidence intervals of ROC curves; ii) investigating how sample sizes influence the properties of the particular methods; iii) recommendations for a general comparison of decision making algorithms via confidence intervals of ROC curves. To support our conclusions we investigate and demonstrate several approaches for constructing parametric confidence intervals on real data. Thus, we present a non-traditional and reliable way of reporting pattern recognition results using ROC curves with confidence intervals.

Optimized Classifier Learning for Face Recognition Performance Boost in Security and Surveillance Applications

“…In this study, in line with the stated objective, we evaluate each method’s influence on the recognition performance of the system described earlier, utilizing the extended database. The cluster description-based methods (namely centroid, weighted centroid, and medoid) and Gaussian mixture model method were also used in our previous research [ 11 , 20 ].…”

“…The cluster description-based methods and the GMM method were also used in our previous research [ 11 , 20 ]. In the subsequent sections ( Section 4.1 and Section 4.3 ), we provide a comparison of the recognition performances of these methods via ROC curves ( Figure 3 , Figure 4 and Figure 5 and Figure 7 , Figure 8 and Figure 9 ) on datasets A1, B, and the newly added A2.…”

“…As demonstrated in these sections ( Section 4.1 and Section 4.3 ), the QIM method outperforms all presented cluster description-based and cluster estimation-based methods, from which the centroid achieved the best results and, thus, was selected as the benchmark. (Further information can be found in [ 11 , 13 , 20 ]).…”