Discovering discriminative action parts from mid-level video representations

Raptis, Michalis; Kokkinos, Iasonas; Soatto, Stefano

doi:10.1109/cvpr.2012.6247807

Cited by 211 publications

(239 citation statements)

References 29 publications

Supporting

Mentioning

234

Contrasting

Order By: Relevance

“…Figure 1), in order to build a hierarchical model of the motion content of a video. This is in contrast to existing approaches [39] that view videos as a bag of clusters. We introduce a corresponding tree representation of actions, called BOF-tree.…”

Section: Introductioncontrasting

confidence: 41%

“…Wang and Mori [50] use tracking and a Hidden Conditional Random Field (HCRF) to learn a discriminative model of latent parts for frame-by-frame recognition. Closest to our work, Raptis et al [39] extract clusters of long-term trajectories and learn a latent model over a fixed number of parts. Their approach has a cubic time complexity in the number of trajectories, relies on bounding box annotations, and uses only a fixed small subset of clusters for all videos.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Recognizing activities with cluster-trees of tracklets

Gaidon¹,

Harchaoui²,

Schmid³

2012

Procedings of the British Machine Vision Conference 2012

View full text Add to dashboard Cite

We address the problem of recognizing complex activities, such as pole vaulting, which are characterized by the composition of a large and variable number of different spatio-temporal parts. We represent a video as a hierarchy of mid-level motion components. This hierarchy is a data-driven decomposition specific to each video. We introduce a divisive clustering algorithm that can efficiently extract a hierarchy over a large number of local trajectories. We use this structure to represent a video as an unordered binary tree. This tree is modeled by nested histograms of local motion features. We provide an efficient positive definite kernel that computes the structural and visual similarity of two tree decompositions by relying on models of their edges. Contrary to most approaches based on action decompositions, we propose to use the full hierarchical action structure instead of selecting a small fixed number of parts. We present experimental results on two recent challenging benchmarks that focus on complex activities and show that our kernel on per-video hierarchies allows to efficiently discriminate between complex activities sharing common action parts. Our approach improves over the state of the art, including unstructured activity models, baselines using other motion decomposition algorithms, graph matching, and latent models explicitly selecting a fixed number of parts.

show abstract

Section: Introductioncontrasting

confidence: 41%

Section: Introductionmentioning

confidence: 99%

Recognizing activities with cluster-trees of tracklets

Gaidon¹,

Harchaoui²,

Schmid³

2012

Procedings of the British Machine Vision Conference 2012

View full text Add to dashboard Cite

show abstract

“…These motion features are then used in an SVM for performing action recognition. Raptis et al [36] discuss another trajectory clustering approach for obtaining mid-level descriptions of the most significant action components useful for recognition. However, we compare our results with our feature detectors in a bag-of-features SVM framework.…”

Section: Related Workmentioning

confidence: 99%

Action recognition using global spatio-temporal features derived from sparse representations

Somasundaram

Cherian

Morellas

et al. 2014

Computer Vision and Image Understanding

View full text Add to dashboard Cite

Recognizing actions is one of the important challenges in computer vision with respect to video data, with applications to surveillance, diagnostics of mental disorders, and video retrieval. Compared to other data modalities such as documents and images, processing video data demands orders of magnitude higher computational and storage resources. One way to alleviate this difficulty is to focus the computations to informative (salient) regions of the video. In this paper, we propose a novel global spatio-temporal selfsimilarity measure to score saliency using the ideas of dictionary learning and sparse coding. In contrast to existing methods that use local spatio-temporal feature detectors along with descriptors (such as HOG, HOG3D, HOF, etc.), dictionary learning helps consider the saliency in a global setting (on the entire video) in a computationally efficient way. We consider only a small percentage of the most salient (least self-similar) regions found using our algorithm, over which spatio-temporal descriptors such as HOG and region covariance descriptors are computed. The ensemble of such block descriptors in a bag-of-features framework provides a holistic description of the motion sequence which can be used in a classification setting. Experiments on several benchmark datasets in video based action classification demonstrate that our approach performs competitively to the state of the art.

show abstract

“…Inspired from the work of [22] and based on a few weak annotations on a sparse set of frames, shown in Figure 8, two types of poselet features, including the HOG descriptors and the BoW features, are used for training the poselet detector. The BoW features, quantized dense descriptors (SIFT [43], histogram of optical flow (HOF) [44], and motion boundaries (HoMB) [45]), are used to augment the HOG descriptors for capturing the motion information of poselets. In this paper, the background information is removed from the poselets by the segmentation scheme, which, in turn, improves both the quality of the poselet models in the learning phase and the recognition accuracy in the testing phase.…”

Section: The Application To Human Activity Recognitionmentioning

confidence: 99%

Unsupervised Object Modeling and Segmentation with Symmetry Detection for Human Activity Recognition

Cheng

Huang

2015

Symmetry

View full text Add to dashboard Cite

Abstract:In this paper we present a novel unsupervised approach to detecting and segmenting objects as well as their constituent symmetric parts in an image. Traditional unsupervised image segmentation is limited by two obvious deficiencies: the object detection accuracy degrades with the misaligned boundaries between the segmented regions and the target, and pre-learned models are required to group regions into meaningful objects. To tackle these difficulties, the proposed approach aims at incorporating the pair-wise detection of symmetric patches to achieve the goal of segmenting images into symmetric parts. The skeletons of these symmetric parts then provide estimates of the bounding boxes to locate the target objects. Finally, for each detected object, the graphcut-based segmentation algorithm is applied to find its contour. The proposed approach has significant advantages: no a priori object models are used, and multiple objects are detected. To verify the effectiveness of the approach based on the cues that a face part contains an oval shape and skin colors, human objects are extracted from among the detected objects. The detected human objects and their parts are finally tracked across video frames to capture the object part movements for learning the human activity models from video clips. Experimental results show that the proposed method gives good performance on publicly available datasets.

show abstract

Discovering discriminative action parts from mid-level video representations

Cited by 211 publications

References 29 publications

Recognizing activities with cluster-trees of tracklets

Recognizing activities with cluster-trees of tracklets

Action recognition using global spatio-temporal features derived from sparse representations

Unsupervised Object Modeling and Segmentation with Symmetry Detection for Human Activity Recognition

Contact Info

Product

Resources

About