3D separable convolutional neural network for dynamic hand gesture recognition

Hu, Zhongxu; Hu, Youmin; Liu, Jie; Wu, Bo; Han, Dongmin; Kurfess, Thomas R.

doi:10.1016/j.neucom.2018.08.042

Cited by 63 publications

(31 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A 3D CNN model allows features to be extracted in both spatial and temporal dimensions by performing 3D convolution. This enables motion information encoded in various frames to be captured [20]. Traditional classification approaches have also been utilised.…”

Section: Related Workmentioning

confidence: 99%

Gesture Recognition Intermediary Robot for Abnormality Detection in Human Activities

Yahaya

Lotfi

Mahmud

et al. 2019

2019 IEEE Symposium Series on Computational Intelligence (SSCI)

View full text Add to dashboard Cite

The world ageing population is increasing, giving rise to research targeted towards improving the quality of life and promoting the independent living of older adults. Detecting abnormalities in the daily activities of the older adults is relevant since abnormalities can be an early sign of health decline, prompting for the need for intervention. Current approaches to abnormality detection involve modelling the usual behavioural routine of the individual as a baseline and comparing subsequent behaviour to the baseline to detect abnormalities. This approach is prone to errors and not flexible since it does not take into account changes in human behavioural routine. Training is usually performed on pre-existing data making the abnormality detection model non-adaptive to new incoming data. An intermediary can be incorporated to enable model predictions to be communicated to humans for verification of the detected anomalies. This paper proposes a gesture recognition approach for facilitating interaction between humans and a robot intermediary. A model capable of recognising hand gestures corresponding to affirmations and denials is implemented. Preliminary evaluation shows that the proposed gesture recognition approach has the potential of being utilised in an assistive robot intermediary.

show abstract

Section: Related Workmentioning

confidence: 99%

Gesture Recognition Intermediary Robot for Abnormality Detection in Human Activities

Yahaya

Lotfi

Mahmud

et al. 2019

2019 IEEE Symposium Series on Computational Intelligence (SSCI)

View full text Add to dashboard Cite

show abstract

“…Convex hull Open CV function was used to detect a number of defects (concavities) on hand and stored into the defect array. Based upon a number of defects, finger count was determined [20]. Kinect V2 to recognize different human hand gestures.…”

Section: Real-time Hand Gesture Recognition Using Kinect V2mentioning

confidence: 99%

“…For data augmentation, they use reverse ordering of the frames and horizontal mirroring, partial elastic deformation, and temporal elastic deformation. 3D convolution process is decomposed into two parts: depth-wise and point-wise [20]. However, due to the decomposition, the network is deepened, which lead to gradient dispersion and renders the network difficult to train.…”

Section: Real-time Hand Gesture Recognition Using 3d Convolutional Nementioning

confidence: 99%

Performance Improvement of Data Fusion Based Real-Time Hand Gesture Recognition by Using 3-D Convolution Neural Networks With Kinect V2

Chandrasekhar¹

2019

IKM

View full text Add to dashboard Cite

Hand gesture recognition is one of the most active areas of research in computer vision. It provides an easy way to interact with a machine without using any extra devices. Hand gestures are natural and intuitive communication way for the human being to interact with his environment. In this paper, we propose Data Fusion Based Real-Time Hand Gesture Recognition using 3-D Convolutional Neural Networks and Kinect V2. To achieve the accurate segmentation and tracking with Kinect V2. Convolution neural network to improve the validity and robustness of the system. Based on the experimental results, the proposed model is accurate, robust and performance with very low processor utilization. The performance of our proposed system in real life application, which is controlling various devices using Kinect V2.

show abstract

“…Multiple modality inputs are used to further improve the performance of these CNN-based models. Depth [ 10 , 14 ] and optical flow [ 11 , 15 ] are the most common complements to the RGB images. However, the depth modality needs an extra sensor to capture accurate information from the user’s hands.…”

Section: Introductionmentioning

confidence: 99%

Improving Real-Time Hand Gesture Recognition with Semantic Segmentation

Benitez-Garcia

Prudente-Tixteco

Castro-Madrid

et al. 2021

Sensors

View full text Add to dashboard Cite

Hand gesture recognition (HGR) takes a central role in human–computer interaction, covering a wide range of applications in the automotive sector, consumer electronics, home automation, and others. In recent years, accurate and efficient deep learning models have been proposed for real-time applications. However, the most accurate approaches tend to employ multiple modalities derived from RGB input frames, such as optical flow. This practice limits real-time performance due to intense extra computational cost. In this paper, we avoid the optical flow computation by proposing a real-time hand gesture recognition method based on RGB frames combined with hand segmentation masks. We employ a light-weight semantic segmentation method (FASSD-Net) to boost the accuracy of two efficient HGR methods: Temporal Segment Networks (TSN) and Temporal Shift Modules (TSM). We demonstrate the efficiency of the proposal on our IPN Hand dataset, which includes thirteen different gestures focused on interaction with touchless screens. The experimental results show that our approach significantly overcomes the accuracy of the original TSN and TSM algorithms by keeping real-time performance.

show abstract

3D separable convolutional neural network for dynamic hand gesture recognition

Cited by 63 publications

References 12 publications

Gesture Recognition Intermediary Robot for Abnormality Detection in Human Activities

Gesture Recognition Intermediary Robot for Abnormality Detection in Human Activities

Performance Improvement of Data Fusion Based Real-Time Hand Gesture Recognition by Using 3-D Convolution Neural Networks With Kinect V2

Improving Real-Time Hand Gesture Recognition with Semantic Segmentation

Contact Info

Product

Resources

About