Learning From Web Videos for Event Classification

Abstract

Abstract-Traditional approaches for classifying event videos rely on a manually curated training dataset. While this paradigm has achieved excellent results on benchmarks such as TrecVid multimedia event detection (MED) challenge datasets, it is restricted by the effort involved in careful annotation. Recent approaches have attempted to address the need for annotation by automatically extracting images from the web, or generating queries to retrieve videos. In the former case, they fail to exploit additional cues provided by video data, while in the latter, they still require some manual annotation to generate relevant queries. We take an alternate approach in this paper, leveraging the synergy between visual video data and the associated textual metadata, to learn event classifiers without manually annotating any videos. Specifically, we first collect a video dataset with queries constructed automatically from textual description of events, prune irrelevant videos with text and video data, and then learn the corresponding event classifiers. We evaluate this approach in the challenging setting where no manually annotated training set is available, i.e., EK0 in the TrecVid challenge, and show state-of-the-art results on MED 2011 and 2013 datasets.

Win-Fail Action Recognition

“…Visual concept learning: We find that works by Binder et al[1], Zhou et al[56], and Chesneau et al[4] are closest to ours. While[1,56] focus on recognizing more complex visual concepts, beyond objects in image domain, we introduce win-fail recognition in the video domain for deeper human action understanding.…”

“…While[1,56] focus on recognizing more complex visual concepts, beyond objects in image domain, we introduce win-fail recognition in the video domain for deeper human action understanding. Chesneau et al[4] address recognizing concepts like 'Birthday Party,' 'Grooming an Animal,' and 'Unstuck a Vehicle' in web videos. However, these concepts do not have large intra-class variance like ours, and are less complex and challenging than ours.…”

Self-Supervised Representation Learning for Videos by Segmenting via Sampling Rate Order Prediction

“…Self-supervised video representation learning methods utilize the correspondence between multiple data streams so that the generated video representation can put the correlation of various modalities of data into consideration. Nicolas et al [37] automatically collect training set from Web videos according to the given textual description and establish the mapping between the textual description and video representation while our method does not require a lot of textual description. Mahendran et al [22] design an auxiliary task based on the correlation verification of RGB video frames and optical flow.…”