Classification of video events using 4-dimensional time-compressed motion features

Abstract

Among the various types of semantic concepts modeled, events pose the greatest challenge in terms of computational power needed to represent the event and accuracy that can be achieved in modeling it. We introduce a novel low-level visual feature that summarizes motion in a shot. This feature leverages motion vectors from MPEG-encoded video, and aggregates local motion vectors over time in a matrix, which we refer to as a motion image. The resulting motion image is representative of the overall motion in a video shot, having compressed the temporal dimension while preserving spatial ordering. Building motion models using this feature permits us to combine the power of discriminant modeling with the dynamics of the motion in video shots that cannot be accomplished by building generative models over a time series of motion features from multiple frames in the video shot. Evaluation of models built using several motion image features in the TRECVID 2005 dataset shows that use of this novel motion feature results an average improvement in concept detection performance by 140% over existing motion features. Furthermore, experiments also reveal that when this motion feature is combined with static feature representations of a single keyframe from the shot such as color and texture features, the fused detection results in an improvement between 4 to 12% over the fusion across the static features alone. Categories and Subject Descriptors General TermsAlgorithms, Performance, Design, Experimentation, Verification.

Automatic semantic video annotation in wide domain videos based on similarity and commonsense knowledgebases

“…A good SVM shots classifier, based on motion feature alone, was suggested in [4]. Bertini et al [12] have built a system that learns events rules from Ontology.…”

“…While in areas like surveillance it is considered initially as one action such as "object1 moves from the left to the right", these actions form "Interest events" in later stages combined with background information like forbidden zones [3]. However, in other areas, the term "Event" holds the wide meanings of concept (such as car, mountain, walking or sport) [4]; or sometimes it is used as a shot class (such as a replay or a break in a football game) [5].…”

Video event detection using motion relativity and visual relatedness

“…In [12], motion vectors are extracted from MPEG encoded videos and compressed to form a motion image. SVM is then used for event recognition.…”

“…to detect and recognize events of user interest from different modalities such as video streams, audio and texts. A lot of efforts have been put to event-based video analysis including unusual event detection [2,4,34,35], action classification [6,10,11,18,23,30,32], and event recognition [9,12,17,31].…”

“…Motion is an important cue in describing event evolution. Recently, various motion features have been developed to capture motion information in the sequence such as motion histogram [6] and motion vector map [12]. However, the existing features are still weak in event-based concept detection because: i) most features only consider one of the two aspects, i.e.…”