HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences

Oreifej, Omar; Liu, Zicheng

doi:10.1109/cvpr.2013.98

Cited by 854 publications

(895 citation statements)

References 22 publications

Supporting

Mentioning

868

Contrasting

Unclassified

Order By: Relevance

“…Some approaches segment the actor, for example using the Kinect's user mask (Gorelick et al 2007;Li et al 2010;Cheng et al 2012). This enables complex "volumetric" descriptions of the actor's body over time (Yang et al 2012;Wang et al 2012;Vieira et al 2012;Oreifej and Liu 2013). However, for "in the wild" action recognition it remains challenging to segment the actor reliably, due to noisy 3D data, cluttered environments, and scenes containing multiple people.…”

Section: Related Workmentioning

confidence: 99%

Hollywood 3D: What are the Best 3D Features for Action Recognition?

2016

View full text Add to dashboard Cite

Action recognition "in the wild" is extremely challenging, particularly when complex 3D actions are projected down to the image plane, losing a great deal of information. The recent growth of 3D data in broadcast content and commercial depth sensors, makes it possible to overcome this. However, there is little work examining the best way to exploit this new modality. In this paper we introduce the Hollywood 3D benchmark, which is the first dataset containing "in the wild" action footage including 3D data. This dataset consists of 650 stereo video clips across 14 action classes, taken from Hollywood movies. We provide stereo calibrations and depth reconstructions for each clip. We also provide an action recognition pipeline, and propose a number of specialised depth-aware techniques including five interest point detectors and three feature descriptors. Extensive tests allow evaluation of different appearance and depth encoding schemes. Our novel techniques exploiting this depth allow us to reach performance levels more than triple those of the best baseline algorithm using only appearance information. The benchmark data, code and calibrations are all made available to the community.

show abstract

Section: Related Workmentioning

confidence: 99%

Hollywood 3D: What are the Best 3D Features for Action Recognition?

2016

View full text Add to dashboard Cite

show abstract

“…The use of 3D data has removed the need for this step in much recent work recognising actions in constrained environments, due to the simplicity of segmenting the actor (for example by using the Kinect's user mask) [19,4,8]. This enables complex "volumetric" descriptions of the actors body over time [34,32,31,23]. However, for "in the wild" action recognition this is not the case as it generally remains impossible to segment the actor reliably, due to noisy 3D data, cluttered environments, and scenes containing multiple people.…”

Section: Related Workmentioning

confidence: 99%

Natural Action Recognition Using Invariant 3D Motion Encoding

Hadfield

Lebeda

Bowden

2014

Computer Vision – ECCV 2014

View full text Add to dashboard Cite

Abstract. We investigate the recognition of actions "in the wild" using 3D motion information. The lack of control over (and knowledge of) the camera configuration, exacerbates this already challenging task, by introducing systematic projective inconsistencies between 3D motion fields, hugely increasing intra-class variance. By introducing a robust, sequence based, stereo calibration technique, we reduce these inconsistencies from fully projective to a simple similarity transform. We then introduce motion encoding techniques which provide the necessary scale invariance, along with additional invariances to changes in camera viewpoint.On the recent Hollywood 3D natural action recognition dataset, we show improvements of 40% over previous state-of-the-art techniques based on implicit motion encoding. We also demonstrate that our robust sequence calibration simplifies the task of recognising actions, leading to recognition rates 2.5 times those for the same technique without calibration. In addition, the sequence calibrations are made available.

show abstract

“…This is why the use of such devices is restricted only in indoor application scenarios with important constraints imposed during the acquisition of visual data (e.g., static cameras). Activity recognition in the wild, however, is actually a different problem, concerning recognition scenarios significantly more demanding than the restricted experimental setups typically employing Kinect [14], with completely different suitable algorithmic solutions than those used in simple action / gesture recognition under constraints (e.g., [15]). …”

Section: Related Workmentioning

confidence: 99%

Exploiting stereoscopic disparity for augmenting human activity recognition performance

Mademlis

Iosifidis

Tefas

et al. 2015

Multimed Tools Appl

View full text Add to dashboard Cite

General rightsThis document is made available in accordance with publisher policies. Please cite only the published version using the reference above. Full terms of use are available: http://www.bristol.ac.uk/pure/about/ebr-terms Abstract-This work investigates several ways to exploit scene depth information, implicitly available through the modality of stereoscopic disparity in 3D videos, with the purpose of augmenting performance in the problem of recognizing complex human activities in natural settings. The standard state-of-the-art activity recognition algorithmic pipeline consists in the consecutive stages of video description, video representation and video classification. Multimodal, depth-aware modifications to standard methods are being proposed and studied, both for video description and for video representation, that indirectly incorporate scene geometry information derived from stereo disparity. At the description level, this is made possible by suitably manipulating video interest points based on disparity data. At the representation level, the followed approach represents each video by multiple vectors corresponding to different disparity zones, resulting in multiple activity descriptions defined by disparity characteristics. In both cases, a scene segmentation is thus implicitly implemented, based on the distance of each imaged object from the camera during video acquisition. The investigated approaches are flexible and able to cooperate with any monocular low-level feature descriptor. They are evaluated using a publicly available activity recognition dataset of unconstrained stereoscopic 3D videos, consisting in extracts from Hollywood movies, and compared both against competing depth-aware approaches and a state-of-the-art monocular algorithm. Quantitative evaluation reveals that some of the examined approaches achieve state-of-the-art performance.

show abstract

HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences

Cited by 854 publications

References 22 publications

Hollywood 3D: What are the Best 3D Features for Action Recognition?

Hollywood 3D: What are the Best 3D Features for Action Recognition?

Natural Action Recognition Using Invariant 3D Motion Encoding

Exploiting stereoscopic disparity for augmenting human activity recognition performance

Contact Info

Product

Resources

About