Actions as space-time shapes

Abstract

Abstract-Human action in video sequences can be seen as silhouettes of a moving torso and protruding limbs undergoing articulated motion. We regard human actions as three-dimensional shapes induced by the silhouettes in the space-time volume. We adopt a recent approach [14] for analyzing 2D shapes and generalize it to deal with volumetric space-time action shapes. Our method utilizes properties of the solution to the Poisson equation to extract space-time features such as local space-time saliency, action dynamics, shape structure, and orientation. We show that these features are useful for action recognition, detection, and clustering. The method is fast, does not require video alignment, and is applicable in (but not limited to) many scenarios where the background is known. Moreover, we demonstrate the robustness of our method to partial occlusions, nonrigid deformations, significant changes in scale and viewpoint, high irregularities in the performance of an action, and low-quality video.

Action categorization with modified hidden conditional random field

“…Those silhouettes are extracted from an already known background. This simple procedure follows exactly the same way as [3]. Note there are more robust methods which can be used for silhouettes extraction of video sequences, e.g.…”

“…Investigation of those methods beyonds the scope of this paper. Instead, we used the binary sequences directly available from [3]. 1 We found that for binary images sequences, simple inter-frame differencing method performs well in our task.…”

“…In general, there are two key elements in modelling human actions [13]: local appearance and temporal dependencies. To date, silhouettebased action recognition has been popular [13,23,20,3], i.e. an action is represented by a series of human body shapes.…”

“…an action is represented by a series of human body shapes. A silhouette is usually extracted by the estimation of the background, or given a known fixed background [3]. Other feature representations of action include space-time interest points [19,14], optical flow [7], motion template [4], and space-time volumes [25], shape context from still images [2,24], etc.…”

Hollywood 3D: What are the Best 3D Features for Action Recognition?

“…In the past, benchmark datasets such as KTH (Schuldt et al 2004), Weizmann (Blank et al 2005) or Kinect based datasets (Li et al 2010;Cheng et al 2012) have been invaluable in providing comparative benchmarks to examine how competing approaches perform in action recognition and detection. However, these staged datasets now routinely have reported performance rates over 90 %, suggesting that they are reaching the end of their service to the community.…”