Supervised Learning and Codebook Optimization for Bag-of-Words Models

Abstract

In this paper, we present a novel approach for supervised codebook learning and optimization for bag-ofwords models. This type of models is frequently used in visual recognition tasks like object class recognition or human action recognition. An entity is represented as a histogram of codewords, which are traditionally clustered with unsupervised methods like k-means or random forests and then classified in a supervised way. We propose a new supervised method for joint codebook creation and class learning, which learns the cluster centers of the codebook in a goal-directed way using the class labels of the training set. As a result, the codebook is highly correlated to the recognition problem, leading to a more discriminative codebook. We propose two different learning algorithms, one based on error backpropagation and the other based on cluster label reassignment. We apply the proposed method to human action recognition from video sequences and evaluate it on the KTH data set, reporting very promising results. The proposed technique allows us to improve the discriminative power of an unsupervised learned codebook or to keep the discriminative power while decreasing the size of the learned codebook, thus decreasing the computational complexity due to the nearest neighbor search.

Spectral Clustering using Optimized Bag-of-Features

“…The methods of the second category tie the classifier and the codebook learning and rely on the classifier's decisions optimize the codebook. In [22], [43], and [26], max-margin formulations are used to learn the codebook, while in [16] a multilayer perceptron (MLP) is used to backpropagate the error to the dictionary. In [47] multiple dictionaries with complementary discriminative information are learned.…”

Image Classification Based on Histogram Intersection Kernel

“…Bag-of-Words (BOW) model was used to express a document (or sentence) as a histogram of frequency of words, ignoring the order of these words in natural language processing firstly [10]. It was used to perform the classification of computer images by Li [11].This model can learn characteristic labels of scenes without human intervention in the training database.…”

Action recognition in videos

“…We recently introduced a fully supervised way for jointly learning the dictionary and the prediction model, which makes the method more discriminant [6].…”