AEkNN: An AutoEncoder kNN–Based Classifier With Built-in Dimensionality Reduction

Abstract

High dimensionality, i.e. data having a large number of variables, tends to be a challenge for most machine learning tasks, including classification. A classifier usually builds a model representing how a set of inputs explain the outputs. The larger is the set of inputs and/or outputs, the more complex would be that model. There is a family of classification algorithms, known as lazy learning methods, which does not build a model. One of the best known members of this family is the kNN algorithm. Its strategy relies on searching a set of nearest neighbors, using the input variables as position vectors and computing distances among them. These distances loss significance in high-dimensional spaces. Therefore kNN, as many other classifiers, tends to worse its performance as the number of input variables grows.In this work AEkNN, a new kNN-based algorithm with built-in dimensionality reduction, is presented. Aiming to obtain a new representation of the data, having a lower dimensionality but with more informational features, AEkNN internally uses autoencoders. From this new feature vectors the computed distances should be more significant, thus providing a way to choose better neighbors. A experimental evaluation of the new proposal is conducted, analyzing several configurations and comparing them against the classical kNN algorithm. The obtained conclusions demonstrate that AEkNN offers better results in predictive and runtime performance.

Revisiting Bayesian Autoencoders with MCMC

“…Table 2 provides the Bayesian autoencoder topology that is used in the experiments described in this section. The autoencoder is trained, and the compressed or reduced dataset is used with another machine learning model for training to get classification accuracy, analogously to [85,86,87,88,89,90]. Note that we consider two topologies for Swiss Roll: the starred variant does not pass the information through large layers but reduces it directly to two dimensions.…”

Galaxy Image Classification Based on Citizen Science Data: A Comparative Study

“…AEs have been proposed for diverse tasks related to feature fusion [32] and dimensionality reduction to facilitate the learning of canonical classifiers [33]. In astronomy, AEs show a great potential for the processing and storage of large 2 https://www.kaggle.com/c/galaxy-zoo-the-galaxy-challenge datasets of images.…”

“…Recent state-of-the-art CNNs are usually composed of a very large number of layers [30] when dealing with challenging image classification problems [31]. Alternatively, DL can also be used to extract features of an image by means of autoencoders (AEs) [32], which have also been proposed to ease the learning of standard classifiers [33]. Whereas CNNs often need to learn the image features from scratch using a large amount of labelled data, AEs enable the encapsulation of the FE process for a particular problem without any need of labels, which can be advantageous for the classification of big collections of images and the use of other kind of classifiers [33].…”

Real-Time User Identification and Behavior Prediction Based on Foot-Pad Recognition

“…In other words, if the difference between the features is not large, the accuracy is lowered because the number of errors with TLS is small. K-NN is also susceptible to the curse of dimensionality: if dimensional reduction is applied, higher accuracy will result [35,36].…”