Articulated motion discovery using pairs of trajectories

Abstract

We propose an unsupervised approach for discovering characteristic motion patterns in videos of highly articulated objects performing natural, unscripted behaviors, such as tigers in the wild. We discover consistent patterns in a bottom-up manner by analyzing the relative displacements of large numbers of ordered trajectory pairs through time, such that each trajectory is attached to a different moving part on the object. The pairs of trajectories descriptor relies entirely on motion and is more discriminative than state-of-the-art features that employ single trajectories. Our method generates temporal video intervals, each automatically trimmed to one instance of the discovered behavior, and clusters them by type (e.g., running, turning head, drinking water). We present experiments on two datasets: dogs from YouTube-Objects and a new dataset of National Geographic tiger videos. Results confirm that our proposed descriptor outperforms existing appearance-and trajectory-based descriptors (e.g., HOG and DTFs) on both datasets and enables us to segment unconstrained animal video into intervals containing single behaviors.

Behavior Discovery and Alignment of Articulated Object Classes from Unstructured Video

“…As some sequences lack pauses between different, but related behaviors (e.g., from walking to running), we also partition based on periodic motion. For this we use time-frequency analysis, as periodic motion patterns like walking, running, or licking typically generate peaks in the frequency domain (examples available on our website Del Pero et al 2015b).…”

“…7.4). We publicly released this data at Del Pero et al (2015b), where we also provide foreground masks for each shot computed using Papazoglou and Ferrari (2013).…”

“…It is fully automatic: the class label is the only supervision we require. Videos with more behaviors are available at Del Pero et al (2015b).…”

“…A preliminary version of this work appeared at CVPR 2015 (Del Pero et al 2015a) covering the behavior discovery stage. In this journal paper, we introduce the spatial alignment stage and present a more extensive experimental evaluation.…”

Discovering thematic visual objects in unconstrained videos

“…The existing action co-localization works have mainly focused on two scenarios, i.e., co-localization in pairs of videos [18,38,97] and weakly supervised action co-localization with video level labels [20,21,92,93,108,134,143]. Few of these works have considered a fully unconstrained scenario like us, i.e., the numbers and types of common actions are unknown in advance and each video may contain zero, one or several common actions.…”