Spatial extensions to bag of visual words

Abstract

The Bag of Visual Words (BoV) paradigm has successfully been applied to image content analysis tasks such as image classification and object detection. The basic BoV approach overlooks spatial descriptor distribution within images. Here we describe spatial extensions to BoV and experimentally compare them in the VOC2007 benchmark image category detection task. In particular, we compare two ways for tiling images geometrically: soft tiling approach-proposed hereand the traditional hard tiling technique. The experiments also address two methods of fusing information from several tilings of the images: post-classifier fusion and fusion on the level of a SVM kernel.The experiments confirm that the performance of a BoV system can be greatly enhanced by taking the descriptors' spatial distribution into account. The soft tiling technique performs well even with a single tiling mask, whereas multimask fusion is necessary for good category detection performance in case of hard tiling. The evaluated fusion mechanisms performed approximately equally well.

Towards Efficient Learning of Optimal Spatial Bag-of-Words Representations

“…For example, Viitaniemi et al observed that manually designed tilings achieve reasonable improvement over the SPM on the Pascal VOC dataset [24]. Similar observations have been confirmed on other datasets.…”

“…However, the problem is still tractable given the reasonable masks including the commonly used masks in the literature [12,24,30,21]. The numbers of all possible set partitions, tilings and equal tilings on different masks are listed in Table 1, where the Parameter column lists the parameters used in generating the masks 1 , e.g.…”

“…The SPM assumes that the spatial BoW representation is independent of data. However, evidence has shown that manually defined representations [24,21,7,30,13] considering salient spatial layouts outperform the predefined BoW representation on many problems.…”

Nearest-Neighbor based Metric Functions for indoor scene recognition

“…The final spatial pyramid kernel is implemented as concatenating weighted histograms of all features at all sub-regions. The traditional bag-of-visual words scheme discards any spatial information; hence many methods utilizing this concept also introduce different spatial extensions [7,24].…”

Approximate Image Matching using Strings of Bag-of-Visual Words Representation

“…In (Sharma and Jurie, 2011), Sharma et al propose to learn the best discriminative grid splitting optimizing a given classification task. In (Viitaniemi and Laaksonen, 2009), Viitaniemi et al compare techniques of soft tiling and hard tiling. Furthermore, some works propose to learn or adapt weights rather than using fixed ones, as in (Harada et al, 2011).…”