Optimal Classifier Model Status Selection Using Bayes Boundary Uncertainty

Ha, David; Delattre, Émilie; Tomotoshi, Yuya; Senda, Mamoru; Watanabe, H.; Katagiri, Shigeru; Ohsaki, Miho

doi:10.1109/mlsp.2018.8516976

Cited by 4 publications

(2 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Experimental results for the method showed its fundamental utility, but these results were not so helpful due to the insufficient implementation of procedures such as finding near-boundary samples, which are defined below, and calculating the entropy. We also applied our method to a task of optimally setting the width of a Gaussian kernel for Support Vector Machine (SVM), while making some improvements to the implementation [20,21]. The experimental results [21] showed its utility more clearly than did the previous experiment [19], but there remained issues to be improved.…”

Section: Introductionmentioning

confidence: 99%

“…We also applied our method to a task of optimally setting the width of a Gaussian kernel for Support Vector Machine (SVM), while making some improvements to the implementation [20,21]. The experimental results [21] showed its utility more clearly than did the previous experiment [19], but there remained issues to be improved. For example, the improved implementation provided suboptimal classifier statuses for some difficult datasets; furthermore, even if the concept basically worked well, it was somewhat heuristically implemented, and thus its theoretical validity was not sufficiently supported.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Practical Method Based on Bayes Boundary-Ness for Optimal Classifier Parameter Status Selection

Tomotoshi

Senda

et al. 2019

J Sign Process Syst

Self Cite

View full text Add to dashboard Cite

We propose a novel practical method for finding the optimal classifier parameter status corresponding to the Bayes error (minimum classification error probability) through the evaluation of estimated class boundaries from the perspective of Bayes boundary-ness. While traditional methods approach classifier optimality from the angle of minimization of the estimated classification error probabilities, we approach it from the angle of optimality of the estimated classification boundaries. The optimal classification boundary consists solely of uncertain samples, whose posterior probability is equal for the two classes separated by the boundary. We refer to this essential characteristic of the boundary as "Bayes boundaryness", and use it to measure how optimal the estimated boundary is. Our proposed method achieves the optimal parameter status using the training data only once, in contrast to such traditional methods as Cross-Validation (CV), which demand separate validation data and often require a number of repetitions of training and validation. Moreover, it can be directly applied to any type of classifier, and potentially to any type of sample. In this paper, we first elaborate on our proposed method that implements the Bayes boundary-ness with an entropy-based uncertainty measure. Next, we analyze the mathematical characteristics of the uncertainty measure adopted. Finally, we evaluate the method through a systematic experimental comparison with CV-based Bayes boundary estimation, which is known to be highly reliable in the Bayes error estimation. From the analysis, we rigorously show the theoretical validity of our adopted uncertainty measure. Moreover, from the experiment, we successfully demonstrate that our method can closely approximate the CV-based Bayes boundary estimate and its corresponding classifier parameter status with only a single-shot training over the data in hand.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%