3D SMoSIFT: three-dimensional sparse motion scale invariant feature transform for activity recognition from RGB-D videos

Wan, Jun; Ruan, Qiuqi; Li, Wei; An, Gaoyun; Zhao, Ruizhen

doi:10.1117/1.jei.23.2.023017

Cited by 45 publications

(22 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3D MoSIFT [16], 3D EMoSIFT [6] and 3D SMoSIFT [1] are derived from MoSIFT using RGB-D data. 3D MoSIFT and 3D EMoSIFT adopt a similar strategy to detect initial interest points.…”

Section: Related Workmentioning

confidence: 99%

“…The single or combined application of 3D SMoSIFT, HOG, HOF and MBH feature descriptors achieves excellent performance in one-shot learning gesture recognition, which has been demonstrated by some state-of-the-art approaches [1,9,10,11,35,36,37], and is also widely used for human activity recognition [15,30,38]. In this paper, 3D SMoSIFT, HOG, HOF and MBH feature descriptors are concatenated to represent gestures.…”

Section: Feature Descriptormentioning

confidence: 99%

“…Vision-based gesture recognition is a critical interface for non-intrusive human-robot interaction (HRI) systems, for which many natural and convenient recognition methods have been proposed [1,2,3]. In a traditional HRI system, a user usually needs to memorize a predefined gesture language or a set of instructions before interaction.…”

Section: Introductionmentioning

confidence: 99%

“…These features should represent well the between-class differences and within-class similarities of gestures [1,5]. Some recently published approaches use the color or depth information to detect spatiotemporal interest points (STIPs), i.e., keypoints, from which to build the feature descriptors.…”

Section: Introductionmentioning

confidence: 99%

“…Then, Histogram of Oriented Gradients (HOG) [14], Histogram of Optical Flow (HOF) [15], HOG/HOF [15] and Viewpoint Feature Histogram Camera Roll Histogram (VFHCRH) feature descriptors are calculated for each keypoint. The 3D motion scale invariant feature transform (3D MoSIFT) [16], 3D enhanced MoSIFT (3D EMoSIFT) [6] and 3D sparse MoSIFT (3D SMoSIFT) [1] spatiotemporal feature approaches, which are extensions of the MoSIFT method [17] using RGB-D data, can be referred to as 3D MoSIFT-based methods. All the 3D MoSIFT-based methods detect the initial interest points in whole frames.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Adaptive Local Spatiotemporal Features from RGB-D Data for One-Shot Learning Gesture Recognition

Lin

Ruan

et al. 2016

Sensors

View full text Add to dashboard Cite

Noise and constant empirical motion constraints affect the extraction of distinctive spatiotemporal features from one or a few samples per gesture class. To tackle these problems, an adaptive local spatiotemporal feature (ALSTF) using fused RGB-D data is proposed. First, motion regions of interest (MRoIs) are adaptively extracted using grayscale and depth velocity variance information to greatly reduce the impact of noise. Then, corners are used as keypoints if their depth, and velocities of grayscale and of depth meet several adaptive local constraints in each MRoI. With further filtering of noise, an accurate and sufficient number of keypoints is obtained within the desired moving body parts (MBPs). Finally, four kinds of multiple descriptors are calculated and combined in extended gradient and motion spaces to represent the appearance and motion features of gestures. The experimental results on the ChaLearn gesture, CAD-60 and MSRDailyActivity3D datasets demonstrate that the proposed feature achieves higher performance compared with published state-of-the-art approaches under the one-shot learning setting and comparable accuracy under the leave-one-out cross validation.

show abstract

“…3D MoSIFT [16], 3D EMoSIFT [6] and 3D SMoSIFT [1] are derived from MoSIFT using RGB-D data. 3D MoSIFT and 3D EMoSIFT adopt a similar strategy to detect initial interest points.…”

Section: Related Workmentioning

confidence: 99%