Practical elimination of near-duplicates from web video search

Abstract

Current web video search results rely exclusively on text keywords or user-supplied tags. A search on typical popular video often returns many duplicate and near-duplicate videos in the top results. This paper outlines ways to cluster and filter out the nearduplicate video using a hierarchical approach. Initial triage is performed using fast signatures derived from color histograms. Only when a video cannot be clearly classified as novel or nearduplicate using global signatures, we apply a more expensive local feature based near-duplicate detection which provides very accurate duplicate analysis through more costly computation. The results of 24 queries in a data set of 12,790 videos retrieved from Google, Yahoo! and YouTube show that this hierarchical approach can dramatically reduce redundant video displayed to the user in the top result set, at relatively small computational cost.

Large-scale near-duplicate web video search: Challenge and opportunity

“…The dictionary is generated by clustering 1,185,698 keypoints extracted from 3,000 keyframes randomly sampled from the dataset of [1]. We employ DoG [6] for keypoint detection and PSIFT [9] for feature description.…”

“…A rough statistic, as indicated in [1], shows that more than 65, 000 videos have been uploaded on video sharing web site YouTube daily. It is believed that this number is still increasing with fast speed.…”

“…These videos were collected from YouTube, Google and Yahoo!. The first time we collected the videos was in year 2006 [1], where these…”

Fast Content-Based Mining of Web2.0 Videos

“…In [15], near-duplicate frames are identified in the TRECVID 2004 video corpus, but 16 seconds are required for searching 150 frames in 10 minutes of video (600 frames). An application of CBVCD to the elimination of video duplicates in Web search is proposed in [14]; global descriptors help separating the least similar videos, then local descriptors allow to refine duplicate detection. However, several minutes are required for returning the top 10 answers (among the 600 preliminary results) to a keyword-based query, which is too long for performing the query-dependent online processing we need.…”

Challenges and techniques for effective and efficient similarity search in large video databases

“…Therefore, an important problem now faced by these video sharing sites is how to automatically perform accurate and fast similarity search for an incoming video clip against its huge database, to avoid copyright violation. Meanwhile, since the retrieval efficiency will be hampered if a large number of search results are essentially almost-identical, database purge also contributes to high-quality ranking for video search results [34].…”