Two streams Recurrent Neural Networks for Large-Scale Continuous Gesture Recognition

Chai, Xiujuan; Liu, Zhipeng; Yin, Fang; Liu, Zhuang; Chen, Xilin

doi:10.1109/icpr.2016.7899603

Cited by 69 publications

(52 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Others have presented neural network architectures to address the problem of gesture recognition directly as a whole. Examples include the Two Streams Recurrent Neural Network proposed by Chai et al [19] or the Recurrent 3D Convolutional Neural Network (R3DCNN) by Molchanov et al [20].…”

Section: Related Workmentioning

confidence: 99%

Gesture Recognition in RGB Videos Using Human Body Keypoints and Dynamic Time Warping

Schneider

Memmesheimer

Kramer

et al. 2019

RoboCup 2019: Robot World Cup XXIII

View full text Add to dashboard Cite

Gesture recognition opens up new ways for humans to intuitively interact with machines. Especially for service robots, gestures can be a valuable addition to the means of communication to, for example, draw the robot's attention to someone or something. Extracting a gesture from video data and classifying it is a challenging task and a variety of approaches have been proposed throughout the years. This paper presents a method for gesture recognition in RGB videos using OpenPose to extract the pose of a person and Dynamic Time Warping (DTW) in conjunction with One-Nearest-Neighbor (1NN) for time-series classification. The main features of this approach are the independence of any specific hardware and high flexibility, because new gestures can be added to the classifier by adding only a few examples of it. We utilize the robustness of the Deep Learning-based OpenPose framework while avoiding the data-intensive task of training a neural network ourselves. We demonstrate the classification performance of our method using a public dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

Gesture Recognition in RGB Videos Using Human Body Keypoints and Dynamic Time Warping

Schneider

Memmesheimer

Kramer

et al. 2019

RoboCup 2019: Robot World Cup XXIII

View full text Add to dashboard Cite

show abstract

“…Chai et al [12] proposed to fuse RGB and depth in a two-stream RNN (2S-RNN) for gesture recognition. They designed a fusion layer for depth and RGB before the LSTM layer.…”

Section: Rnn-based Approachmentioning

confidence: 99%

RGB-D-based human motion recognition with deep learning: A survey

Wang

Ogunbona

et al. 2018

Computer Vision and Image Understanding

325

170

View full text Add to dashboard Cite

Human motion recognition is one of the most important branches of human-centered research activities. In recent years, motion recognition based on RGB-D data has attracted much attention. Along with the development in artificial intelligence, deep learning techniques have gained remarkable success in computer vision. In particular, convolutional neural networks (CNN) have achieved great success for image-based tasks, and recurrent neural networks (RNN) are renowned for sequence-based problems. Specifically, deep learning methods based on the CNN and RNN architectures have been adopted for motion recognition using RGB-D data. In this paper, a detailed overview of recent advances in RGB-D-based motion recognition is presented. The reviewed methods are broadly categorized into four groups, depending on the modality adopted for recognition: RGB-based, depth-based, skeleton-based and RGB+D-based.As a survey focused on the application of deep learning to RGB-D-based motion recognition, we explicitly discuss the advantages and limitations of existing techniques. Particularly, we highlighted the methods of encoding spatialtemporal-structural information inherent in video sequence, and discuss potential directions for future research. body parts, in contrast with the few body parts that involved in gesture. An activity is composed by a sequence of actions. An interaction is a type of motion performed by two actors; one actor is human while the other may be human or an object. This implies that the interaction category will include human-human or human-object interaction."Hugging each other" and "playing guitar" are examples of these two kinds of interaction, respectively. Group activity is the most complex type of activity, and it may be a combination of gestures, actions and interactions. Necessarily, it involves more than two humans and from zero to multiple objects. Examples of group activities would include "two teams playing basketball" and "group meeting".Early research on human motion recognition was dominated by the analysis of still images or videos [2,144,132,99,44,176]. Most of these efforts used color and texture cues in 2D images for recognition. However, the task remains challenging due to problems posed by background clutter, partial occlusion, view-point, lighting changes, execution rate and biometric variation. This challenge remains even with current deep learning approaches [49,4].With the recent development of cost-effective RGB-D sensors, such as Microsoft Kinect TM and Asus Xtion TM , RGB-D-based motion recognition has attracted much attention. This is largely because the extra dimension (depth) is insensitive to illumination changes and includes rich 3D structural information of the scene. Additionally, 3D positions of body joints can be estimated from depth maps [114]. As a consequence, several methods based on RGB-D data have been proposed and the approach has proven to be a promising direction for human motion analysis.Several survey papers have summarized the research on human motion recognition...

show abstract

“…Then, RNN is employed for modeling large-scale temporal dependencies, data fusion and ultimately gesture classification. A multi stream RNN is also proposed by Chai et al (2016) for large scale gesture spotting. Eleni (2015) propose a Convolutional Long Short-Term Memory Recurrent Neural Network (CNNLSTM) able to successfully learn gesture varying in duration and complexity.…”

Section: Temporal Deep Learning Models: Rnn and Lstmmentioning

confidence: 99%

Deep Learning for Action and Gesture Recognition in Image Sequences: A Survey

Asadi-Aghbolaghi

Clapés

Bellantonio

et al. 2017

Gesture Recognition

View full text Add to dashboard Cite

Interest in automatic action and gesture recognition has grown considerably in the last few years. This is due in part to the large number of application domains for this type of technology. As in many other computer vision areas, deep learning based methods have quickly become a reference methodology for obtaining state-of-the-art performance in both tasks. This chapter is a survey of current deep learning based methodologies for action and gesture recognition in sequences of images. The survey reviews both fundamental and cutting edge methodologies reported in the last few years. We introduce a taxonomy that summarizes important aspects of deep learning for approaching both tasks. Details of the proposed architectures, fusion strategies, main datasets, and competitions are reviewed. *. A reduced version of this appeared appeared as: M. Asadi-Aghbolaghi et al. A survey on deep learning based approaches for action and gesture recognition in image sequences. In

show abstract

Two streams Recurrent Neural Networks for Large-Scale Continuous Gesture Recognition

Cited by 69 publications

References 18 publications

Gesture Recognition in RGB Videos Using Human Body Keypoints and Dynamic Time Warping

Gesture Recognition in RGB Videos Using Human Body Keypoints and Dynamic Time Warping

RGB-D-based human motion recognition with deep learning: A survey

Deep Learning for Action and Gesture Recognition in Image Sequences: A Survey

Contact Info

Product

Resources

About