Large Scale Category-Structured Image Retrieval for Object Identification Through Supervised Learning of CNN and SURF-Based Matching

Abstract

In the modern era of Internet, mobile and digital information technology, image retrieval for object identification, just as wine label retrieval from a wine bottle image, has become an important and urgent problem in artificial intelligence. In comparison with the general image retrieval, it is rather challenging because there are a huge number of object identification or brand images which are very similar and difficult to discriminate, and the number of different brand images in the given dataset changes greatly, that is, the samples are strongly unbalanced for these brands. In this paper, we propose a CNN-SURF Consecutive Filtering and Matching (CSCFM) framework for this kind of image retrieval, specifically focalizing on wine label retrieval. In particular, Convolutional Neural Network (CNN) is utilized to filter out the impossible main-brands (manufacturers) for narrowing down the range of retrieval and the Speeded Up Robust Features (SURF) matching is improved by adopting the RANdom SAmple Consensus (RANSAC) mechanism and the modified Term Frequency-Inverse Document Frequency (TF-IDF) distance for the accurate retrieval of the sub-brand (item attribute under the manufacture). The experiments are conducted on a dataset containing approximately 548k images of wine labels with 17, 328 main-brands and 260, 579 subbrands. It is demonstrated by the experimental results that our proposed method can solve the wine label retrieval problem effectively and efficiently. Moreover, our proposed method is further evaluated on two pubic benchmarks of the object identification image retrieval tasks, Oxford Buildings Benchmark (Oxford5k) and the University of Kentucky of Indoor Things Benchmark (UKB), and achieves 88.3% mean average precision and 3.92 N-S score in Oxford5k and UKB, respectively. INDEX TERMS Image retrieval, object identification, wine label retrieval, CNN, SURF descriptor, filter out the impossible main-brands.

Image Retrieval Method Based on Image Feature Fusion and Discrete Cosine Transform

“…The image retrieval performance can be enhanced through feature representation, feature quantization, or sorting method. The CNN-SURF Consecutive Filtering and Matching (CSCFM) framework [74] uses the deep feature representation by CNN to filter out the impossible main-brands for narrowing down the range of retrieval. The Deep Supervised Hashing (DSH) method [75] designs a CNN loss function to maximize the discriminability of the output space by encoding the supervised information from the input image pairs.…”

Content-based image retrieval for fabric images: A survey

“…Global features Suitable for large datasets and both supervised and unsupervised learning [8], [10], [11], [13], [15], [16], [19] Reduces the semantic gap in comparison with traditional methods [8], [15], [16], [21] Can adapt to illumination and background interference [29], [36] Not suitable for sketch or grayscale images [28] Low accuracy for images with many patterns, such as printed fabric or clothing with complex shapes [2], [34], [39] Large feature dimensions [48], [54] High computation time [31], [36], [41] 6. CONCLUSION Feature selection and extraction methods are critical aspects of fabric image retrieval.…”

Dynamic optimization of hessian determinant image pyramid for memory‐efficient and high performance keypoint detection in SURF

“…Recently, most of the studies [12][13][14][15][16] on SURF have been related to combination with other algorithms [12][13][14][15] or application-specific modification [16] without reducing high memory usage and heavy computations in SURF itself. To be specific, [12] improved SURF by adding feedback mechanism based on CNN in order to reduce distortion in image mosaic.…”