A Neural Network Based Distance Function for the k-Nearest Neighbor Classifier

Abstract

Due to its simplicity and intuitivity, the k-nearest neighbor method is one of the most commonly used technique to address different classification problems. However, to apply such a classification technique, a distance metric is to be considered to define a certain distance in the feature space. Usually classic norms such as Minkowski, Mahalanobis, etc. are used, but even though they can be applied to all type of feature spaces, there is a lack of specificity of these methods for the data in use. Our goal is to learn the distance function rather than providing an explicit one, by exploiting the specificity of the data. Several neural networks are trained to estimate the distance between two patterns, adapting the weight of the networks accordingly. For the experimental setup, we considered three well-known benchmark, publicly available data collections: MNIST digit data, Opt-digits data, and the Lampung character collection -for Arabic handwritten digits and Indonesian handwritten Lampung characters, respectively. The results of k-nearest neighbor considering network based estimated distance between the handwritten digits/characters proved to be very promising.