Large-Scale Video Retrieval via Deep Local Convolutional Features

Abstract

In this paper, we study the challenge of image-to-video retrieval, which uses the query image to search relevant frames from a large collection of videos. A novel framework based on convolutional neural networks (CNNs) is proposed to perform large-scale video retrieval with low storage cost and high search efficiency. Our framework consists of the key-frame extraction algorithm and the feature aggregation strategy. Specifically, the key-frame extraction algorithm takes advantage of the clustering idea so that redundant information is removed in video data and storage cost is greatly reduced. The feature aggregation strategy adopts average pooling to encode deep local convolutional features followed by coarse-to-fine retrieval, which allows rapid retrieval in the large-scale video database. The results from extensive experiments on two publicly available datasets demonstrate that the proposed method achieves superior efficiency as well as accuracy over other state-of-the-art visual search methods.

Real-time detection of partial video copy on TV workstation

“…Several features have been investigated to characterize the frames for the video copy detection such as the CNN [5], SURF [6] and the BRIEF [7] features. These features can handle major degradations but have a little real-time ability.…”

“…The frames with a significant visual change are selected as key-frames. The distortion could be measured with deep features [5], the NCC difference [8] or the maximum entropy [10]. We propose here a key-frame selection method minimizing the distortion between the reference and altered videos.…”

“…As discussed in section III-B, the standard approach is to select the key-frames having significant visual changes in the reference videos. This could be obtained with the CNN features [5], the NCC difference [8] or the maximum entropy [10]. For a further comparison, we have considered too the SURF and BRIEF features used for frame matching [6], [7].…”

“…A near 1 to 3 key-frames are selected per video. Competitive detections are obtained at a slightly gap F 0.98 with the CNN features [5] and the max entropy [10] for selection, considering a same level complexity and response time. As a general trend, our system is able to detect in real-time a near 80K of reference videos from 24 channels with a strong accuracy.…”

Image and video face retrieval with query image using convolutional neural network features

“…We can apply standard features for image retrieval [17]- [20] by processing each frame as an independent image. More recent works showed that is possible to use CNN for feature extraction when working on videos [21], [22].…”