IR Feature Embedded BOF Indexing Method for Near-Duplicate Video Retrieval

“…Video-level retrieval methods encode the videos in video-level, and search for the k-nearest neighbors for the video-level feature of the query video in the embedding space. Various frame feature aggregation methods [3,5,18,19,20,21] have been used to obtain a single video-level representation. Liong et al [22] propose temporal pooling layer to aggregate the successive frames by the means of average pooling.…”

Section: Video-level Retrieval Methodsmentioning

confidence: 99%

Self-supervised Video Retrieval Transformer Network

He,

Pan,

Tang

et al. 2021

Preprint

View full text Add to dashboard Cite

Content-based video retrieval aims to find videos from a large video database that are similar to or even nearduplicate of a given query video. It plays an important role in many video related applications, including copyright protection, recommendation, filtering and etc.. Video representation and similarity search algorithms are crucial to any video retrieval system. To derive effective video representation, most video retrieval systems require a large amount of manually annotated data for training, making it costly inefficient. In addition, most retrieval systems are based on frame-level features for video similarity searching, making it expensive both storage wise and search wise. We propose a novel video retrieval system, termed SVRTN, that effectively addresses the above shortcomings. It first applies self-supervised training to effectively learn video representation from unlabeled data to avoid the expensive cost of manual annotation. Then, it exploits transformer structure to aggregate frame-level features into clip-level to reduce both storage space and search complexity. It can learn the complementary and discriminative information from the interactions among clip frames, as well as acquire the frame permutation and missing invariant ability to support more flexible retrieval manners. Comprehensive experiments on two challenging video retrieval datasets, namely FIVR-200K and SVD, verify the effectiveness of our proposed SVRTN method, which achieves the best performance of video retrieval on accuracy and efficiency.

show abstract

“…For efficient video similarity measurement. Most retrieval methods have a straightforward motivation aggregating the local frame-level features into clip-level or even video-level representations, such as global vectors [15,25], hash codes [8,9,19], Bag-of-Words (BoW) [2,14,16], and video similarity is measured by distances of aggregated representations. However, the aggregated representations are too coarse to cover abundant fine-grained information and can't be used to partial segment localization.…”

Section: Related Workmentioning

confidence: 99%

Video Similarity and Alignment Learning on Partial Video Copy Detection

Han

Tang

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Existing video copy detection methods generally measure video similarity based on spatial similarities between key frames, neglecting the latent similarity in temporal dimension, so that the video similarity is biased towards spatial information. There are methods modeling unified video similarity in an end-to-end way, but losing detailed partial alignment information, which causes the incapability of copy segments localization. To address the above issues, we propose the Video Similarity and Alignment Learning (VSAL) approach, which jointly models spatial similarity, temporal similarity and partial alignment. To mitigate the spatial similarity bias, we model the temporal similarity as the mask map predicted from frame-level spatial similarity, where each element indicates the probability of frame pair lying right on the partial alignments. To further localize partial copies, the step map is learned from the spatial similarity where the elements indicate extending directions of the current partial alignments on the spatial-temporal similarity map. Obtained from the mask map, the start points extend out into partial optimal alignments following instructions of the step map. With the similarity and alignment learning strategy, VSAL achieves the state-of-the-art 𝐹 1 -score on VCDB core dataset. Furthermore, we construct a new benchmark of partial video copy detection and localization by adding new segment-level annotations for FIVR-200k dataset, where VSAL also achieves the best performance, verifying its effectiveness in more challenging situations. Our project is publicly available at https://pvcd-vsal.github.io/vsal/. CCS CONCEPTS• Computing methodologies → Matching; Visual content-based indexing and retrieval.

show abstract

IR Feature Embedded BOF Indexing Method for Near-Duplicate Video Retrieval

Cited by 20 publications

References 40 publications

A Deep Analysis of Image Based Video Searching Techniques

A Deep Analysis of Image Based Video Searching Techniques

Self-supervised Video Retrieval Transformer Network

Video Similarity and Alignment Learning on Partial Video Copy Detection

Contact Info

Product

Resources

About