Benchmarking unsupervised near-duplicate image detection

Abstract

Unsupervised near-duplicate detection has many practical applications ranging from social media analysis and web-scale retrieval, to digital image forensics. It entails running a threshold-limited query on a set of descriptors extracted from the images, with the goal of identifying all possible near-duplicates, while limiting the false positives due to visually similar images. Since the rate of false alarms grows with the dataset size, a very high specificity is thus required, up to 1 − 10 −9 for realistic use cases; this important requirement, however, is often overlooked in literature. In recent years, descriptors based on deep convolutional neural networks have matched or surpassed traditional feature extraction methods in content-based image retrieval tasks. To the best of our knowledge, ours is the first attempt to establish the performance range of deep learning-based descriptors for unsupervised near-duplicate detection on a range of datasets, encompassing a broad spectrum of near-duplicate definitions. We leverage both established and new benchmarks, such as the Mir-Flick Near-Duplicate (MFND) dataset, in which a known ground truth is provided for all possible pairs over a general, large scale image collection. To compare the specificity of different descriptors, we reduce the problem of unsupervised detection to that of binary classification of near-duplicate vs. not-near-duplicate images. The latter can be conveniently characterized using Receiver Operating Curve (ROC). Our findings in general favor the choice of fine-tuning deep convolutional networks, as opposed to using off-the-shelf features, but differences at high specificity settings depend on the dataset and are often small. The best performance was observed on the MFND benchmark, achieving 96% sensitivity at a false positive rate of 1.43 × 10 −6 .

Single- and Cross-Modality Near Duplicate Image Pairs Detection via Spatial Transformer Comparing CNN

“…Through such a setup, pseudo-Siamese is able to provide more flexibility than restricted Siamese, while restricted Siamese is more efficient in training [ 36 ]. Furthermore, DeepRet [ 2 , 37 ] extends Siamese into triplet Siamese with the Triple Loss to capture the slight difference between images and applies region of interest (ROI) pooling to deeply learn the images.…”

“…Thus, there are many near-duplicate images existing among the massive data. Near-duplicate images are typically considered as images containing the same scene or objects [ 1 , 2 , 3 ], but in various viewpoints, illumination conditions and camera settings etc, or images obtained through reediting of the original image [ 4 ], including but not limited to changing contrast, tone, cropping, rotating and watermarking. Automatic near-duplicate image pair detection by using computer vision and pattern recognition technology has been attracting wide attention recently, as it has great potential values in the application of image copyright violations detection, fake image detection, management of device hardware storage, and autonomous vehicle driving.…”

Near-Duplicate Image Detection System Using Coarse-to-Fine Matching Scheme Based on Global and Local CNN Features

“…Recently, some researchers have focused on training deep learning models to extract global image features. Lia et al [34] evaluated a set of CNN-learned descriptors and concluded that the features learned from fine-tuned CNNs perform better than the off-the-shelf features. Shervin et al [35] gave a summary of promising works that use deep learning-based models for biometric recognition.…”

“…Recently, some researchers have focused on training deep learning models to extract image features [34][35][36][37]. However, since these methods fail to sufficiently consider the influence of background clutter and partial occlusion on the final representation, the extracted global CNN features are not robust enough to combat these attacks.…”

Media Forensics Challenge image provenance evaluation and state-of-the-art analysis on large-scale benchmark datasets

“…However, the problem of IPA is not intensively studied so far in the literature. The research areas that are related to IPA are near duplicate detection [22,23], and image splicing detection [24][25][26][27][28][29]. Most of these works are designed to classify whether a candidate image is a near duplicate to a given query image.…”