Recognizing and Tracking Human Action

Abstract

Human activity can be described as a sequence of 3D body postures. The traditional approach to recognition and 3D reconstruction of human activity has been to track motion in 3D, mainly using advanced geometric and dynamic models. In this paper we reverse this process. View based activity recognition serves as an input to a human body location tracker with the ultimate goal of 3D reanimation in mind. We demonstrate that specific human actions can be detected from single frame postures in a video sequence. By recognizing the image of a person's posture as corresponding to a particular key frame from a set of stored key frames, it is possible to map body locations from the key frames to actual frames. This is achieved using a shape matching algorithm based on qualitative similarity that computes point to point correspondence between shapes, together with information about appearance. As the mapping is from fixed key frames, our tracking does not suffer from the problem of having to reinitialise when it gets lost. It is effectively a closed loop. We present experimental results both for recognition and tracking for a sequence of a tennis player.

Temporal motion models for monocular and multiview 3D human body tracking

“…Detection, involving pose recognition from individual frames, has become increasingly popular in recent research [20][21][22][23][24] but requires large numbers of training poses to be effective. Tracking involves pose inference at one time instant given state information (e.g., pose) from previous time instants.…”

Multiple Tree Models for Occlusion and Spatial Constraints in Human Pose Estimation

“…For example, Toyama & Blake [21] use 2D exemplars for people tracking. Mori & Malik [11] and Sullivan & Carlsson [19] address the pose estimation problems as 2D template matching using pre-stored exemplars upon which joint locations have been marked. In order to deal with the complexity due to variations of pose and clothing, Shakhnarovich et al [14] adopt a bruteforce search, using a variant of locality sensitive hashing for speed.…”

“…However, if the structure of the graphical model is not a tree, one has to use loopy belief propagations. In that case, the convolution trick is no longer valid, since the message stored at a node is no longer in a simple form that allows the derivation of (19) to go through. This further justifies the advantage of using tree-structured models.…”

Human Action Recognition Based on Spatio-temporal Features

“…A detailed survey of action recognition techniques has been presented by Gavrila [1]. The researchers have used space-time features to identify the specific action [2,3]. Computer vision scientists have tried motion based techniques [4,5,6] as any action is associated with some motion.…”