Recognize Human Activities from Partially Observed Videos

Cao, Yu; Barrett, Daniel Paul; Barbu, Andrei; Narayanaswamy, Siddharth; Yu, Haonan; Michaux, Aaron; Lin, Yuewei; Dickinson, Sven; Siskind, Jeffrey Mark; Wang, Song

doi:10.1109/cvpr.2013.343

Cited by 175 publications

(161 citation statements)

References 22 publications

Supporting

Mentioning

161

Contrasting

Order By: Relevance

“…These include running the actual released code for C2 [24], Action Bank [57], Stacked ISA [36], and VHTK [44]. We also obtained the code for Cao's method [6], Cao's reimplementation [6] of Ryoo's method [56], and Retinotopic [3] from the authors. We also employ a number of other recent methods, including Dense Trajectories [72,73], Improved Trajectories [74], C3D [69], and the methods of Simonyan and Zisserman [60], Ng et al [46], and Xu et al [80].…”

Section: Baseline Experimentsmentioning

confidence: 99%

“…The dataset used for the THUMOS Challenge differs from the LCA dataset in several ways. First, the THUMOS Challenge uses trimmed videos from 6 Comparison between machine-human and human-human intercoder agreement on the LCA dataset, using the evaluation metric from the THUMOS Challenge [25], comparing against a single human annotator: cbushman UCF101 [65] as the training set; only the validation and test sets involve untrimmed videos. The LCA dataset is partitioned into five sets of videos for training and test; each set of videos consists of untrimmed videos.…”

Section: Related Workmentioning

confidence: 99%

“…These include KTH (6 classes, [58]), Weizmann (10 classes, [4]), CMU Soccer (7 classes, [9]), CMU Crowded (5 classes, [27]), UCF Sports (9 classes, [53]), UR ADL (10 classes, [44]), UM Gesture (14 classes, [38]), UCF Youtube (11 classes, [41]), Hollywood-1 (8 classes, [35]), Hollywood-2 (12 classes, [43]), MultiKTH (6 classes, [70]), MSR (3 classes, [86]), and TRECVID (10 classes, [63]). These datasets contain short clips, each depicting one of a small number of classes (3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14). Several more recent datasets also contain short clips, each depicting a single action, but with a larger number of action classes: UCF50 (50 classes, [52]), HMDB51 (51 classes, [33]), and UCF101 (101 classes, [65]).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Collecting and annotating the large continuous action dataset

Barrett

et al. 2016

Machine Vision and Applications

Self Cite

View full text Add to dashboard Cite

We make available to the community a new dataset to support action recognition research. This dataset is different from prior datasets in several key ways. It is significantly larger. It contains streaming video with long segments containing multiple action occurrences that often overlap in space and/or time. All actions were filmed in the same collection of backgrounds so that background gives little clue as to action class. We had five humans to replicate the annotation of temporal extent of action occurrences labeled with their classes and measured a surprisingly low level of intercoder agreement. Baseline experiments show that recent state-of-the-art methods perform poorly on this dataset. This suggests that this will be a challenging dataset to foster advances in action recognition research. This manuscript serves to describe the novel content and characteristics of the LCA dataset, present the design decisions made when filming the dataset, document the novel methods employed to annotate the dataset, and present the results of our baseline experiments.

show abstract

Section: Baseline Experimentsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Collecting and annotating the large continuous action dataset

Barrett

et al. 2016

Machine Vision and Applications

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [33], a probabilistic framework for activity recognition from partially observed videos is introduced. Sparse coding is used to estimate posterior probabilities with bag of visual words representation.…”

Section: Probabilistic Approachesmentioning

confidence: 99%

Intent Understanding Using an Activation Spreading Architecture

2015

View full text Add to dashboard Cite

Abstract:In this paper, we propose a new approach for recognizing intentions of humans by observing their activities with a color plus depth (RGB-D) camera. Activities and goals are modeled as a distributed network of inter-connected nodes in an Activation Spreading Network (ASN). Inspired by a formalism in hierarchical task networks, the structure of the network captures the hierarchical relationship between high-level goals and low-level activities that realize these goals. Our approach can detect intentions before they are realized and it can work in real-time. We also extend the formalism of ASNs to incorporate contextual information into intent recognition. We further augment the ASN formalism with special nodes and synaptic connections to model ordering constraints between actions, in order to represent and handle partial-order plans in our ASN. A fully functioning system is developed for experimental evaluation. We implemented a robotic system that uses our intent recognition to naturally interact with the user. Our ASN based intent recognizer is tested against three different scenarios involving everyday activities performed by a subject, and our results show that the proposed approach is able to detect low-level activities and recognize high-level intentions effectively in real-time. Further analysis shows that contextual and partial-order ASNs are able to discriminate between otherwise ambiguous goals.

show abstract

“…Their experiments are focused on trajectories prediction, but the proposal has been presented for general situations. Cao et al presented in [5] a sparse coding usage and subsamples of the sequence to predict posterior activities for partially observed sequences. Uddin et al proposed in [24] a Human Activity Prediction (HPA) system which uses spanning-trees to predict and recognize activities.…”

mentioning

confidence: 99%

A Novel Prediction Method for Early Recognition of Global Human Behaviour in Image Sequences

Azorín-López

Saval-Calvo

Fuster-Guilló

et al. 2015

Neural Process Lett

View full text Add to dashboard Cite

Human behaviour recognition has been, and still remains, a challenging problem that involves different areas of computational intelligence. The automated understanding of people activities from video sequences is an open research topic in which the computer vision and pattern recognition areas have made big efforts. In this paper, the problem is studied from a prediction point of view. We propose a novel method able to early detect behaviour using a small portion of the input, in addition to the capabilities of it to predict behaviour from new inputs. Specifically, we propose a predictive method based on a simple representation of trajectories of a person in the scene which allows a high level understanding of the global human behaviour. The representation of the trajectory is used as a descriptor of the activity of the individual. The descriptors are used as a cue of a classification stage for pattern recognition purposes. Classifiers are trained using the trajectory representation of the complete sequence. However, partial sequences are processed to evaluate the early prediction capabilities having a specific observation time of the scene. The experiments have been carried out using the three different dataset of the CAVIAR database taken into account the behaviour of an individual. Additionally, different classic classifiers have been used for experimentation in order to evaluate the robustness of the proposal. Results confirm the high accuracy of the proposal on the early recognition of people behaviours.

show abstract

Recognize Human Activities from Partially Observed Videos

Cited by 175 publications

References 22 publications

Collecting and annotating the large continuous action dataset

Collecting and annotating the large continuous action dataset

Intent Understanding Using an Activation Spreading Architecture

A Novel Prediction Method for Early Recognition of Global Human Behaviour in Image Sequences

Contact Info

Product

Resources

About