RRV: A Spatiotemporal Descriptor for Rigid Body Motion Recognition

Abstract

The motion behaviors of a rigid body can be characterized by a six degrees of freedom motion trajectory, which contains the 3-D position vectors of a reference point on the rigid body and 3-D rotations of this rigid body over time. This paper devises a rotation and relative velocity (RRV) descriptor by exploring the local translational and rotational invariants of rigid body motion trajectories, which is insensitive to noise, invariant to rigid transformation and scale. The RRV descriptor is then applied to characterize motions of a human body skeleton modeled as articulated interconnections of multiple rigid bodies. To show the descriptive ability of our RRV descriptor, we explore its potentials and applications in different rigid body motion recognition tasks. The experimental results on benchmark datasets demonstrate that our RRV descriptor learning discriminative motion patterns can achieve superior results for various recognition tasks.

3D Skeletal Gesture Recognition via Sparse Coding of Time-Warping Invariant Riemannian Trajectories

“…And also including two TSRVF related methods, the body part features with SRV and k-nearest neighbors clustering [4] (SRV-KNN), TSRVF on Kendall's shape space [2] (Kendall-TSRVF). The methods in second group are based on classic feature representations, like histogram of 3D joints (HOJ3D) [23], Eigen-Joints [24], actionlet ensemble (Actionlet) [20], histogram of oriented 4D normals (HON4D) [16], rotation and relative velocity with DTW (RVV+DTW) [7], naive Bayes nearest neighbor (NBNN) [21]. The last group including seven deep learning methods, namely the convolutional neural network based ModDrop (CNN) [15], HMM with deep belief network (HMM-DBN) [22], LSTM [9], hierarchical recurrent neural network (HBRNN) [5], spatio-temporal LSTM with trust gates (ST-LSTM-TG) [13], and global context-aware attention LSTM (GCA-LSTM) [14].…”

“…Compared to RGB data, skeletal data is robust to varied background and is invariant to camera view-point. In the past decade, a considerable number of 3D skeleton-based recognition methods [23,24,19,4,3,2,16,7,21,20,22,15,5,13,14] have been proposed. Although there have been significant advancements in this area, accurate recognition of the human gesture in unconstrained settings still remains challenging.…”

“…Although LSTM is a powerful framework for modeling sequential data, it is still arduous to learn the information of the entire sequence with many sub-events. In fact, the most common solution to temporal dynamics is the Dynamic Time Warping (DTW) [19,7], which needs to choose a nominal temporal alignment, and then all sequences of a category are warped to that alignment. However, the performance of DTW is highly depends on the selection of a reference, which is commonly computed by experience.…”

Hidden States Exploration for 3D Skeleton-Based Gesture Recognition

“…To account for temporal dynamics, a common treatment is the dynamic time warping (DTW), as adopted in [35,8,46]. DTW resorts to finding an optimal temporal alignment, then warps all sequences in the same category to a corresponding template.…”

“…Over the last few years, various 3D skeleton-based models have been developed for gesture recognition, ranging from feature representations [44,45,35,7,3,1,48,27,26,8] to various forms of parametric approaches [36,37,32,28,22,29,14,15,41,38,46]; and also including many deep learning methods [43,42,25,10,4,49,23,17,19,30,20,13]. Despite the encouraging progress having been made by various studies, accurately recognizing human gestures is still challenging.…”