Simplicity of Kmeans Versus Deepness of Deep Learning: A Case of Unsupervised Feature Learning with Limited Data

Abstract

Abstract-We study a bio-detection application as a case study to demonstrate that Kmeans-based unsupervised feature learning can be a simple yet effective alternative to deep learning techniques for small data sets with limited intra-as well as inter-class diversity. We investigate the effect on the classifier performance of data augmentation as well as feature extraction with multiple patch sizes and at different image scales. Our data set includes 1833 images from four different classes of bacteria, each bacterial culture captured at three different wavelengths and overall data collected during a three-day period. The limited number and diversity of images present, potential random effects across multiple days, and the multi-mode nature of class distributions pose a challenging setting for representation learning. Using images collected on the first day for training, on the second day for validation, and on the third day for testing Kmeans-based representation learning achieves 97% classification accuracy on the test data. This compares very favorably to 56% accuracy achieved by deep learning and 74% accuracy achieved by handcrafted features. Our results suggest that data augmentation or dropping connections between units offers little help for deeplearning algorithms, whereas significant boost can be achieved by Kmeans-based representation learning by augmenting data and by concatenating features obtained at multiple patch sizes or image scales.

Improving short text classification through global augmentation methods

“…Limited data is a challenge both in getting well-labelled data for supervised learning problems [8] and also continues to be a bigger challenge for low-resource languages [5]. Here we present what the effect of the different augmentation schemes are when labelled data is reduced.…”

Hybrid Elman Neural Network and an Invasive Weed Optimization Method for Bridge Defect Recognition

“…Deep learning requires huge training datasets to capture features and build the relationships between the set of independent variables and the dependent variables, which can sometimes be difficult to find and tedious to create. This sometimes causes over-fitting triggered by the presence of small datasets, which may drastically influence the recognition capacity of the machine-learning models ( 45 , 46 ). Another disadvantage of deep learning is that it sometimes induces detection latency as a result of the high processing demands ( 47 ).…”

A scalable attribute-aware network embedding system

“…With the development of unsupervised feature learning techniques [3], deep learning methods proved successful in natural language processing tasks through neural language models [10,33,36]. These models have been used to capture the semantic and syntactic structures of human language [8], and even logical analogies [20], by embedding words as vectors.…”