Human Action Recognition Based on 3D Human Modeling and Cyclic HMMs

Abstract

Human action recognition is used in areas such as surveillance, entertainment, and healthcare. This paper proposes a system to recognize both single and continuous human actions from monocular video sequences, based on 3D human modeling and cyclic hidden Markov models (CHMMs). First, for each frame in a monocular video sequence, the 3D coordinates of joints belonging to a human object, through actions of multiple cycles, are extracted using 3D human modeling techniques. The 3D coordinates are then converted into a set of geometrical relational features (GRFs) for dimensionality reduction and discrimination increase. For further dimensionality reduction, k-means clustering is applied to the GRFs to generate clustered feature vectors. These vectors are used to train CHMMs separately for different types of actions, based on the Baum-Welch re-estimation algorithm. For recognition of continuous actions that are concatenated from several distinct types of actions, a designed graphical model is used to systematically concatenate different separately trained CHMMs. The experimental results show the effective performance of our proposed system in both single and continuous action recognition problems. Keywords I. IntroductionHuman action recognition is a growing topic in video analysis and understanding -one of the most popular areas in the community of computer vision -thanks to its applications in surveillance, entertainment, and healthcare. In surveillance, human action recognition can be used in conjunction with video camera footage to help with the recognition and analysis of human actions. In entertainment, human-computer interaction can be helped to appear more natural via human action recognition, which in turn can help increase the entertainment experience. In healthcare, human action recognition can help detect abnormal gaits or assist in a patient's rehabilitation through an analysis of their actions.However, it is challenging to recognize various human actions due to the high number of degrees of freedom associated with the average human body -namely, variations in human poses; variations in the colors of a person's clothing; changes in lighting and illumination; variations in viewpoints; and frequent self-occlusion. Moreover, the use of monocular video sequences further increases the difficulty for human action recognition.Generally, the two main stages in human action recognition are: the feature extraction and representation stage and the classification stage.In the feature extraction and representation stage, the features or characteristics of video frames, such as silhouette, shape, color, and motion, are extracted and represented in a systematic and efficient way. consists of stacking segmented silhouettes (frame by frame) to form a 3D spatial-temporal shape. In a similar way, Ke and others [2] build STVs, for shape-based matching, from image features that are based on the consecutive silhouettes of objects along a time axis, including spatial-temporal region extraction and region matching. Kim and oth...

Motion-Vector Refinement for Video Error Concealment Using Downhill Simplex Approach

“…After assigning the initial motion vector, the motion vector is fine tuned. Many applications, including computer vision and video processing, have been reported to solve the nonlinear optimization problems by using a downhill simplex technique .…”

An Online Continuous Human Action Recognition Algorithm Based on the Kinect Sensor

“…In [ 38 ], a visual alignment technique named dynamic frame warping is proposed, which performs isolated recognition based on aligning a test sequence with a model sequence. In [ 39 ], a graphical model is designed to systematically concatenate different separately trained cyclic hidden Markov models for continuous action recognition. In [ 40 ], a probabilistic graphical model with substructure transition model and discriminative boundary model is proposed for continuous action recognition.…”

Skin Condition Estimation Using Mobile Handheld Camera

“…The detection of human characteristics is a steadily growing area of research, and has attracted a good deal of interest and effort [1]- [5]. In recent years, the field of application in this area has expanded to include medical purposes (for example, the detection of melanoma) [6]- [10].…”

“…We concentrated on a threshold-based approach owing to its rapidity and reduced complexity. For skin feature extraction, thresholding represents one of the most commonly used techniques [5], [12]- [14], [16]. Otsu [17], [18] proposed a broadly used conventional thresholding method that assumes that an input image has a district bi-modal distribution, classifying images into two classes by finding a threshold that minimizes the intra-class variance of each class.…”