[Invited Paper] Strategies for Searching Video Content with Text Queries or Video Examples

Abstract

(student member) †2 , Shicheng Xu (student member) †1 , Deyu Meng (member) †3 , Zexi Mao (member) †1 , Zhigang Ma (member) †1 , Ming Lin (member) †1 , Xuanchong Li (student member) †1 , Huan Li (member) †1 , Zhenzhong Lan (student member) †1 , Lu Jiang (student member) †1 , Alexander G. Hauptmann (member) †1 , Chuang Gan (student member) †4 , Xingzhong Du (student member) †5 , Xiaojun Chang (student member) †2Abstract The large number of user-generated videos uploaded on to the Internet everyday has led to many commercial video search engines, which mainly rely on text metadata for search. However, metadata is often lacking for user-generated videos, thus these videos are unsearchable by current search engines. Therefore, content-based video retrieval (CBVR) tackles this metadata-scarcity problem by directly analyzing the visual and audio streams of each video. CBVR encompasses multiple research topics, including low-level feature design, feature fusion, semantic detector training and video search/reranking. We present novel strategies in these topics to enhance CBVR in both accuracy and speed under different query inputs, including pure textual queries and query by video examples. Our proposed strategies have been incorporated into our submission for the TRECVID 2014 Multimedia Event Detection evaluation, where our system outperformed other submissions in both text queries and video example queries, thus demonstrating the effectiveness of our proposed approaches.

Content-Based Video Retrieval With Prototypes of Deep Features

“…They showed that deep features were more efficient for CBVR compared to other low-level features. Yu et al [26] also modified the architecture of 2D CNNs and utilized more data to improve the performance capabilities of existing 2D CNNs. Furthermore, they proposed a new feature fusion method to improve both the performance and robustness.…”