Human action recognition based on 3D skeleton part-based pose estimation and temporal multi-resolution analysis

Abstract

International audienceHuman action recognition is a challenging field that have been addressed with many different classification techniques such as SVM or Random Decision Forests and by considering many different kinds of information joints, key poses, joints rotation matrix, angles for example. This paper presents our approach for action recognition that considers only information given by the 3D joints from the skeleton and trains a two stage random forest to classify them. We extract skeletal features by computing all angles between any triplet of joints and all distances between any pair of joints then organizing them into a feature vector for each static pose. Complex dynamic actions are then described by sequences of such feature vectors. We evaluate our approach on the most recent and the largest benchmark, MSRC-12 Kinect Gesture Dataset, and compare our results with the state-of-the-art methods on this dataset

Supervised Learning for Human Action Recognition from Multiple Kinects

“…All those studies show that using skeleton instead of purely RGB data is a good solution. Combining the skeleton with another classifier such as the Random Forest also shows interesting results [11] on the MSR-DailyActivity3D Dataset. In this study the vector representing a moment in the flow of the data is composed of all the coordinate of the joints of the skeleton and all the distances and angles between the joints.…”

“…This vector doesn't contain enough data to fully classify different actions. Similarly to the work done in [11], we have augmented our feature vector by compute and add all possible distances/angles between all possible pair/triplet of joints. This process ended to a feature vector of 3610 values: 3420 angle values followed by 190 distance values.…”

“…In order to preserve the coverage we had with the Diff2 feature vector and the overlapping sliding window. We also define a Middle Diff5 feature vector by calculating the middle 5 position and averaging the Diff2 at these positions [11]. The Figure 4 shows the consistency of each Diff feature vector.…”

“…We can then integrate the data obtained for the final prediction results.The prediction is done thanks to a learning algorithm. In our research, we developed a successful approach based on Random Forest [11].…”

Advances in human action recognition: an updated survey

“…Instead of using relative joint position, Gori et al [16] used all possible joint pairs in the Euclidean distance sense, and track these distances over time to form relation history images of joints. In another approach, Aly Halim et al [27] calculated all possible distances and angles from triplet joints, i.e. a set of three joint locations, which amounts to 190 distances and 3420 angles.…”