Spatial orientations of visual word pairs to improve Bag-of-Visual-Words model

Abstract

This paper presents a novel approach to incorporate spatial information in the bag-ofvisual-words model for category level and scene classification. In the traditional bag-ofvisual-words model, feature vectors are histograms of visual words. This representation is appearance based and does not contain any information regarding the arrangement of the visual words in the 2D image space. In this framework, we present a simple and efficient way to infuse spatial information. Particularly, we are interested in explicit global relationships among the spatial positions of visual words. Therefore, we take advantage of the orientation of the segments formed by Pairs of Identical visual Words (PIW). An evenly distributed normalized histogram of angles of PIW is computed. Histograms produced by each word type constitute a powerful description of intra type visual words relationships. Experiments on challenging datasets demonstrate that our method is competitive with the concurrent ones. We also show that, our method provides important complementary information to the spatial pyramid matching and can improve the overall performance.

Bregman pooling: feature-space local pooling for image classification

“…Our method also outperformed Khan et al [31,32] on all three datasets. However, it is worth mentioning that even with a relatively small feature dimension (smaller codebook) and less dense low-level features, [32] achieved a highly competitive result on 15 Scenes.…”

“…Moreover, it would be interesting, in the future, to test how adopting either (or both) of them affects the classification performance of the proposed method. Lazebnik et al [35] 64.6 ± 0.8 81.4 ± 0.5 -4200, 8400 Yang et al [53] 73.2 ± 0.5 80.2 ± 0.9 40.1 ± 0.9 21504 Wang et al [51] 73.4 -47.7 43008, 86016 Boureau et al [7] a 71.8 ± 1.0 84.1 ± 0.5 -21504 Boureau et al [8] 77.3 ± 0.6 83.3 ± 1.0 41.6 ± 0.6 b 1397760, 365568, 344064 Chatfield et al [10] 76.1 ± 0.6 --84000 Khan et al [31] 67.1 82.5 -5000 Koniusz et al [33] 81.3 ± 0.6 --86016 Wang et al [52] -84.3 ± 0.2 -43008 Fanello et al [19] --47.9 1134592 Khan et al [32] 68 Chatfield et al [11] 77.6 ± 0.1…”

“…Thus, in order to avoid comparing apples to oranges, we break the listed works into four main groups based on what component, of the SP model, each improves. Thus, works are grouped into (1) those that improve the coding step, including works by Yang et al [53], Wang et al [51] and Wang et al [52], (2) those that improve the pooling operator, including works by Yang et al [53] and Koniusz et al [33], (3) those that enrich the spatial information captured by the model, the works by Khan et al [31,32], and finally (4) those that locally pool in the feature space, including works by Boureau et al [8], Fanello et al [19], and ours. Table 1 also includes studies by Boureau et al [7] and Chatfield et al [10], which are two widely cited benchmarking studies that extensively evaluated the model using different combinations of components and parameters.…”

Invariant Image-Based Species Classification of Butterflies and Reef Fish

“…Khan et al [7] proposed to use the angles made by pair-wise identical visual words (PIWs) to add spatial information to the BoVWs model. An image representation is then constructed on these angles by aggregating them in a pair-wise identical words angles histogram (PIWAH).…”

“…The Multiscale SIFT with best performing number of scales is then evaluated along with the other three extraction methods on predefined vocabulary sizes. The best performing variant of SIFT is then used to compare the performances of PIWAH [7] and TIWAH. Each experiment is performed 10 times where in each run the datasets are randomly split into training and test sets.…”