On the use of feature tracks for dynamic concept detection in video

Abstract

This paper proposes the use of feature tracks for the detection of concepts in video, particularly dynamic concepts. Feature tracks are defined as sets of local interest points found in different frames of a video shot that exhibit spatio-temporal and visual continuity, defining a trajectory in the 2D+Time space. The extraction of feature tracks and the selection and representation of an appropriate subset of them allow the generation of a Bag-of-Spatiotemporal-Words model for the shot, which facilitates capturing the dynamics of video content. The experimental evaluation of the proposed approach highlights how the selection of such feature tracks for the definition of the Bag-of-Spatiotemporal-Words model enhances the results of traditional keyframe-based concept detection techniques.

Survey on modeling and indexing events in multimedia

“…The most important is that they fail to capture the conceptual and contextual information conveyed by the multimedia content. -Concept-based indexing: Many works have appeared on combining the aforementioned low-level features with machine learning algorithms in order to achieve the association of content with semantic concepts such as "person", "outdoors", etc., or different actions [60,52]. The content segments can then be retrieved with the use of the detected concepts [74,16].…”