Near-Duplicate Image Detection System Using Coarse-to-Fine Matching Scheme Based on Global and Local CNN Features

Abstract

Due to the great success of convolutional neural networks (CNNs) in the area of computer vision, the existing methods tend to match the global or local CNN features between images for near-duplicate image detection. However, global CNN features are not robust enough to combat background clutter and partial occlusion, while local CNN features lead to high computational complexity in the step of feature matching. To achieve high efficiency while maintaining good accuracy, we propose a coarse-to-fine feature matching scheme using both global and local CNN features for real-time near-duplicate image detection. In the coarse matching stage, we implement the sum-pooling operation on convolutional feature maps (CFMs) to generate the global CNN features, and match these global CNN features between a given query image and database images to efficiently filter most of irrelevant images of the query. In the fine matching stage, the local CNN features are extracted by using maximum values of the CFMs and the saliency map generated by the graph-based visual saliency detection (GBVS) algorithm. These local CNN features are then matched between images to detect the near-duplicate versions of the query. Experimental results demonstrate that our proposed method not only achieves a real-time detection, but also provides higher accuracy than the state-of-the-art methods.

CNN Feature-Based Image Copy Detection with Contextual Hash Embedding

“…(3) Global CNN + Local CNN + CF [3]: This method extracts both the global and local CNN features from the CFMs generated by a pre-trained CNN model-i.e., Alexnet [19]-and match these features between images with a coarse-to-fine (CF) strategy for copy detection.…”

“…Due to the rapid development of Internet technology and the increasing popularity of personal digital camera devices, the amount of digital media (images, audio, and video) grows exponentially on the Internet [1][2][3]. With the help of various image processing tools such as Photoshop, it is very easy for users to modify a copyrighted image (an original image) with a variety of manipulations such as rescaling, rotation, cropping, noise addition, and text addition to produce various kinds of copy versions of the image for illegal use.…”

“…Some earlier works [24][25][26][27] feed an image into a pretrained CNN model and then use the output of the last fully-connected network layer as a global image representation. In some other works [3,23,26], instead of focusing on the features extracted from fully-connected layers, the features extracted from the deep convolutional layers are explored for the tasks of content-based retrieval.…”

“…Generally, the convolutional feature maps (CFMs) are first extracted from the deep convolutional layers of CNNs with an input image, and a pooling strategy such as sum-pooling [23] or max-pooling [28] is usually adopted to aggregate the feature maps into a single image representation. In our previous work [3], instead of only generating a single global representation from each image, we extract both the global and local CNN features from the CFMs and match these features between images with a coarse-to-fine matching strategy for near-duplicate image detection. In another work [29], the spatial-temporal CNN features are generated and matched for video copy detection.…”

Blind Photograph Watermarking with Robust Defocus-Based JND Model

“…In the past few years, some techniques have made some developments such as block cipher [2], encryption [3], and steganography [4]. Especially with the development of the internet, all kinds of multimedia can be searched easily [5][6][7]. It can even be achieved by cross-modal retrieval [8,9].…”