Visual Word Ambiguity

Abstract

This paper studies automatic image classification by modeling soft assignment in the popular codebook model. The codebook model describes an image as a bag of discrete visual words selected from a vocabulary, where the frequency distributions of visual words in an image allow classification. One inherent component of the codebook model is the assignment of discrete visual words to continuous image features. Despite the clear mismatch of this hard assignment with the nature of continuous features, the approach has been successfully applied for some years. In this paper, we investigate four types of soft assignment of visual words to image features. We demonstrate that explicitly modeling visual word assignment ambiguity improves classification performance compared to the hard assignment of the traditional codebook model. The traditional codebook model is compared against our method for five well-known data sets: 15 natural scenes, Caltech-101, Caltech-256, and Pascal VOC 2007/2008. We demonstrate that large codebook vocabulary sizes completely deteriorate the performance of the traditional model, whereas the proposed model performs consistently. Moreover, we show that our method profits in high-dimensional feature spaces and reaps higher benefits when increasing the number of image categories.

Content-Based Retrieval of Focal Liver Lesions Using Bag-of-Visual-Words Representations of Single- and Multiphase Contrast-Enhanced CT Images

“…The BoW model treats an image as a distribution of local descriptors, wherein each descriptor is labeled as a discrete visual prototype. The set containing these prototypes, or visual words, is the so-called visual vocabulary (or dictionary [16], codebook [22]), which is typically obtained by clustering in the feature space of local descriptor. Given a visual vocabulary, an image is represented as a histogram of visual word occurrences on the sampled image patches from the image.…”

“…Unlike hard assignment, assigning a degree of similarity to an image patch (soft assignment) can help in modeling the inherent uncertainty of the image patch while considering the continuous nature of image patches [22]. Soft assignment can be easily incorporated in the BOW model by…”

Complex Events Detection Using Data-Driven Concepts

“…The idea is that we want to find a representation Y ∈ R d , which can map each clip of different modalities from raw signal space to a semantic space, where clips with similar concepts are near with each other. Considering the high diversity of our data, instead of pooling the low-level TICA features spatial-temporally, we use bag of word (BoW) histogram Q ∈ R D by adopting vector quantization (VQ) technique using K-means soft assignment [23].…”

Self-explanatory Sparse Representation for Image Classification

“…Dictionary (also called vocabulary) learning is the key step here. One standard version of vocabulary learning is K-means clustering on image patches combined with hard-or soft-assignment vector quantization (VQ) [7]. Spatial pyramid matching (SPM) is typically incorporated in the pipeline to compensate the loss of spatial information [12].…”

“…With the introduction of kernel techniques, the learned dictionary becomes versatile. For the K-means based scheme, [38] learned a dictionary in the histogram intersection kernel (HIK) space, while [8] learned it in the Gaussian radial basis function (RBF) kernel space. For the sparse representation based scheme, [23] proposed kernel K-SVD and kernel MOD methods.…”