To Aggregate or Not to aggregate: Selective Match Kernels for Image Search

Tolias, Giorgos; Avrithis, Yannis; Jeǵou, Hervé

doi:10.1109/iccv.2013.177

Cited by 190 publications

(164 citation statements)

References 29 publications

Supporting

Mentioning

162

Contrasting

Order By: Relevance

“…Given the sparsity of BoW and the low discriminative ability of visual words, the inverted index and binary signatures are used [13]. The trade-off between accuracy and efficiency is a major influencing factor [20].…”

Section: Categorization Methodologymentioning

confidence: 99%

“…Considering the relatively small computational cost compared with large codebooks (Section 3.4.1), flat k-means can be adopted for codebook generation [85], [20]. It is also shown in [31], [86] that using AKM [12] for clustering also yields very competitive retrieval accuracy.…”

Section: Codebook Generation and Quantizationmentioning

confidence: 99%

“…The improvement over HE has been observed in a number of works, especially from the view of match kernel [20]. To reduce the information loss on the query side, Jain et al [90] propose a vector-to-binary distance comparison.…”

Section: Hamming Embedding and Its Improvementsmentioning

confidence: 99%

“…While the average distance between a word to its neighbors is regularized to be almost constant in [92], the idea of democratizing the contribution of individual embeddings has later been employed in [18]. In [20], Tolias et al show that VLAD and HE share similar natures and propose a new match kernel which trades off between local feature aggregation and feature-to-feature matching, using a similar matching function to [91]. They also demonstrate that using more bits (e.g., 128) in HE is superior to the original 64 bits scheme at the cost of decreased efficiency.…”

Section: Hamming Embedding and Its Improvementsmentioning

confidence: 99%

See 3 more Smart Citations

SIFT Meets CNN: A Decade Survey of Instance Retrieval

Zheng

Yang

Tian

2018

IEEE Trans. Pattern Anal. Mach. Intell.

639

319

View full text Add to dashboard Cite

Abstract-In the early days, content-based image retrieval (CBIR) was studied with global features. Since 2003, image retrieval based on local descriptors (de facto SIFT) has been extensively studied for over a decade due to the advantage of SIFT in dealing with image transformations. Recently, image representations based on the convolutional neural network (CNN) have attracted increasing interest in the community and demonstrated impressive performance. Given this time of rapid evolution, this article provides a comprehensive survey of instance retrieval over the last decade. Two broad categories, SIFT-based and CNN-based methods, are presented. For the former, according to the codebook size, we organize the literature into using large/medium-sized/small codebooks. For the latter, we discuss three lines of methods, i.e., using pre-trained or fine-tuned CNN models, and hybrid methods. The first two perform a single-pass of an image to the network, while the last category employs a patch-based feature extraction scheme. This survey presents milestones in modern instance retrieval, reviews a broad selection of previous works in different categories, and provides insights on the connection between SIFT and CNN-based methods. After analyzing and comparing retrieval performance of different categories on several datasets, we discuss promising directions towards generic and specialized instance retrieval.

show abstract

Section: Categorization Methodologymentioning

confidence: 99%

Section: Codebook Generation and Quantizationmentioning

confidence: 99%

Section: Hamming Embedding and Its Improvementsmentioning

confidence: 99%

Section: Hamming Embedding and Its Improvementsmentioning

confidence: 99%

See 2 more Smart Citations

SIFT Meets CNN: A Decade Survey of Instance Retrieval

Zheng

Yang

Tian

2018

IEEE Trans. Pattern Anal. Mach. Intell.

639

319

View full text Add to dashboard Cite

show abstract

“…Second, during matching verification, the Hamming distance between two binary features can be efficiently calculated via xor operations, while the Euclidean distance between floating-point vectors is very expensive to compute. Previous work of this line includes Hamming Embedding (HE) [1] and its variants [10], [11], which use binary SIFT features for verification. Meanwhile, binary features also include spatial context [12], heterogeneous feature such as color [13], etc.…”

mentioning

confidence: 99%

Coupled Binary Embedding for Large-Scale Image Retrieval

Zheng

Wang

Tian

2014

IEEE Trans. on Image Process.

133

View full text Add to dashboard Cite

Abstract-Visual matching is a crucial step in image retrieval based on the bag-of-words (BoW) model. In the baseline method, two keypoints are considered as a matching pair if their SIFT descriptors are quantized to the same visual word. However, the SIFT visual word has two limitations. First, it loses most of its discriminative power during quantization. Second, SIFT only describes the local texture feature. Both drawbacks impair the discriminative power of the BoW model and lead to false positive matches. To tackle this problem, this paper proposes to embed multiple binary features at indexing level. To model correlation between features, a multi-IDF scheme is introduced, through which different binary features are coupled into the inverted file. We show that matching verification methods based on binary features, such as Hamming embedding, can be effectively incorporated in our framework. As an extension, we explore the fusion of binary color feature into image retrieval. The joint integration of the SIFT visual word and binary features greatly enhances the precision of visual matching, reducing the impact of false positive matches. Our method is evaluated through extensive experiments on four benchmark datasets (Ukbench, Holidays, DupImage, and MIR Flickr 1M). We show that our method significantly improves the baseline approach. In addition, largescale experiments indicate that the proposed method requires acceptable memory usage and query time compared with other approaches. Further, when global color feature is integrated, our method yields competitive performance with the state-of-the-arts.Index Terms-Feature fusion, coupled binary embedding, multi-IDF, image retrieval.

show abstract

Image Pre-Processing

Krig

2014

Computer Vision Metrics

View full text Add to dashboard Cite

This chapter describes the methods used to prepare images for further analysis, including interest point and feature extraction. Some of these methods are also useful for global and local feature description, particularly the metrics derived from transforms and basis spaces. The focus is on image pre-processing for computer vision, so we do not cover the entire range of image processing topics applied to areas such as computational photography and photo enhancements, so we refer the interested reader to various other standard resources in Digital Image Processing and Signal Processing as we go along [4,9,325,326], and we also point out interesting research papers that will enhance understanding of the topics. Note ■ Readers with a strong background in image processing may benefit from a light reading of this chapter. Perspectives on Image Processing Image processing is a vast field that cannot be covered in a single chapter. So why do we discuss image pre-processing in a book about computer vision? The reason is to advance the science of local and global feature description, as image pre-processing is typically ignored in discussions of feature description. Some general image processing topics are covered here in light of feature description, intended to illustrate rather than to proscribe, as applications and image data will guide the image pre-processing stage. Some will argue that image pre-processing is not a good idea, since it distorts or changes the true nature of the raw data. However, intelligent use of image pre-processing can provide benefits and solve problems that ultimately lead to better local and global feature detection. We survey common methods for image enhancements and corrections that will affect feature analysis downstream in the vision pipeline in both favorable and unfavorable ways, depending on how the methods are employed.

show abstract

To Aggregate or Not to aggregate: Selective Match Kernels for Image Search

Cited by 190 publications

References 29 publications

SIFT Meets CNN: A Decade Survey of Instance Retrieval

SIFT Meets CNN: A Decade Survey of Instance Retrieval

Coupled Binary Embedding for Large-Scale Image Retrieval

Image Pre-Processing

Contact Info

Product

Resources

About