The 2021 Image Similarity Dataset and Challenge

Abstract

This paper introduces a new benchmark for largescale image similarity detection. This benchmark is used for the Image Similarity Challenge at NeurIPS'21 (ISC2021). The goal is to determine whether a query image is a modified copy of any image in a reference corpus of size 1 million. The benchmark features a variety of image transformations such as automated transformations, hand-crafted image edits and machine-learning based manipulations. This mimics real-life cases appearing in social media, for example for integrity-related problems dealing with misinformation and objectionable content. The strength of the image manipulations, and therefore the difficulty of the benchmark, is calibrated according to the performance of a set of baseline approaches. Both the query and reference set contain a majority of "distractor" images that do not match, which corresponds to a real-life needle-in-haystack setting, and the evaluation metric reflects that. We expect the DISC21 benchmark to promote image copy detection as an important and challenging computer vision task and refresh the state of the art.

3rd Place: A Global and Local Dual Retrieval Solution to Facebook AI Image Similarity Challenge

“…The measure of effectiveness is micro-average precision (µAP ) across all submitted image pairs, ranked by confidence score, also known as area under the precision-recall curve [4]. See the official website for additional details.…”

“…In recent years, large-scale retrieval has become more and more important and practical. Different from the Google Landmarks Datasets [15] composed of natural landmark pictures, the Facebook AI Image Similarity Challenge 1 provided a new challenging datasets originating from social media platforms, which is a new benchmark for large-scale image similarity detection [4].…”

“…This competition datasets is mainly suitable for manipulative advertising, preventing uploads of graphic violence, copy detection and enforcing copyright protections [4]. The goal is to determine whether a query image is a modified copy of any image in a reference corpus of size 1 million, as shown in Fig.…”

Producing augmentation-invariant embeddings from real-life imagery

“…The Facebook Image Similarity Dataset and Challenge is a benchmark for large-scale image similarity detection [5]. The dataset consists of 1 million reference images and 100 thousand query images (50 thousand for Phase 1 and 50 thousand for Phase 2 of the competition).…”

“…The similarity score for a given reference-query pair is calculated as the negative of the squared Euclidean distance between the embeddings of each pair of images. To measure the overall performance of the model, the competition evaluates the micro Average Precision (µAP ) calculated from the similarity scores [5].…”