Stochastic Multiview Hashing for Large-Scale Near-Duplicate Video Retrieval

Abstract

Near-duplicate video retrieval (NDVR) has been a significant research task in multimedia given its high impact in applications, such as video search, recommendation and copyright protection, etc. In addition to accurate retrieval performance, the exponential growth of online videos has imposed heavy demands on the efficiency and scalability of the existing systems. Aiming at improving both the retrieval accuracy and speed, we propose a novel stochastic multi-view hashing algorithm to facilitate the construction of a large-scale NDVR system. Reliable mapping functions, which convert multiple types of keyframe features, enhanced by auxiliary information such as video-keyframe association and ground truth relevance to binary hash code strings, are learned by maximizing a mixture of the generalized retrieval precision and recall scores. A composite Kullback-Leibler (KL) divergence measure is used to approximate the retrieval scores, which aligns stochastically the neighborhood structures between the original feature and the relaxed hash code spaces. The efficiency and effectiveness of the proposed method are examined using two public near-duplicate video collections, and are compared against various classical and state-of-the-art NDVR systems.

Unsupervised t-Distributed Video Hashing and Its Deep Hashing Extension

“…Sophisticated video indexing techniques have been developed to accelerate the search speed, such as tree-based [8], [27] and hashing [15], [26], [28]. Treebased indexing partitions the video/image representation space from coarse to fine and forms a hierarchical tree structure [29].…”

“…Their underlying optimizations are formulated via KL divergence. To improve the hashing performance, multi-view hashing techniques have also been developed to learn compact and efficient binary hash codes from a mixture of multiple feature views [15], [26], [36], [37]. Examples of unsupervised works relying on multi-view information, include the multiple feature hashing (MFH) [15], which learns hash codes for videos by manually weighting the importance of different types of feature sources, and also the multi-view alignment hashing (MAH) [36], which fuses the alignment representations from multiple sources while preserving the joint distributions.…”

“…Building upon our previous work on multi-view hashing (SMVH) [26], we propose an unsupervised mechanism for constructing a composite similarity structure between keyframes supported by different types of feature representations. To capture the unsupervised similarity information more accurately, we take into account both the continuous relevance estimation based on a Gaussian distribution over pairwise distances and also the discrete neighbor relationships by examining the cardinality of the reciprocal neighbors.…”

“…This provides a robust unsupervised training method for deep hashing models. Compared to the state-of-the-art hashing methods in [15], [26], the proposed one is also easier to optimize, since the utilization of the t-distribution in the construction of the probabilistic model simplifies the gradients of the cost function. The performance of the proposed method is demonstrated with benchmark evaluations and comparisons with state-of-the-art techniques.…”

“…Section II briefly reviews related works. Section III outlines SMVH [26] which is the starting point of the current work. Section IV explains the proposed work on unsupervised similarity construction, t-distributed matching and the unsupervised deep hashing extension.…”