Selection of Classifiers Using Information-Theoretic Criteria

Selection of Multiple Classifiers With No Prior Limit to the Number of Classifiers by Minimizing the Conditional Entropy

“…Under such an assumption, classifiers are ordered according to recognition rates or reliability rates, and then the classifiers are sequentially selected up to the fixed number from the best one. And also, measure of closeness and conditional entropy based on information theory were applied to select a promising classifier set among classifier sets composed of the fixed number of classifiers (Kang, 2005).…”

“…But, a classifier set by the clustering can not guarantee the best performance over others, so the number of selected classifiers still remains an unresolved issue. Various diversity criteria of classifiers are GD (within-set generalization diversity) proposed by Partridge and Yates, Q statistic proposed by Kuncheva et al, CD (compound diversity) proposed by Roli and Giacinto, and mutual information between classifiers proposed by Kang (Kang, 2005;Roli and Giacinto, 2002). And additional diversity criteria of classifier sets used in the clustering to decide the number of classifiers are GDB (between-set generalization diversity) proposed by Partridge and Yates, and diversity function proposed by Roli and Giacinto(Roli and Giacinto, 2002).…”

“…There have been a lot of studies to solve the pattern recognition problems with a multiple classifier system composed of multiple classifiers. The studies on a multiple classifier system have proceeded in both directions of how to combine multiple classifiers (Ho, 2002;Kang and Lee, 1999;Kittler et al, 1998;Saerens and Fouss, 2004;Woods et al, 1997) and how to select multiple classifiers from a classifier pool (Ho, 2002;Kang, 2005;Roli and Giacinto, 2002), because the performance of a multiple classifier system depends on the selected classifiers as well as a combination method. It is well known that a set of classifiers showing only high recognition rates is not superior to other set of classifiers in most cases (Roli and Giacinto, 2002;Woods et al, 1997), furthermore it was also reported that fewer classifiers provided superior results to more classifiers (Woods et al, 1997).…”

“…It is well known that a set of classifiers showing only high recognition rates is not superior to other set of classifiers in most cases (Roli and Giacinto, 2002;Woods et al, 1997), furthermore it was also reported that fewer classifiers provided superior results to more classifiers (Woods et al, 1997). Thus, it is well recognized that the selection of component classifiers in a multiple classifier system is one of very difficult research issues (Ho, 2002;Kang, 2005;Roli and Giacinto, 2002;Woods et al, 1997).…”

“…Previous studies on the selection of multiple classifiers select a classifier set based on the assumption that the number of selected classifiers is fixed in advance (Kang, 2005), or based on the clustering followed the diversity criteria of classifiers in the over-produce and choose paradigm without such assumption (Roli and Giacinto, 2002). The selected classifier set is used in a multiple classifier system and then multiple classifiers in the set are combined for fusing their classification results.…”