Improving Continuous Hand Gesture Detection and Recognition from Depth Using Convolutional Neural Networks

Tran, Thanh-Hai; Do, Van-Hieu

doi:10.1007/978-981-16-2094-2_10

Cited by 6 publications

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a practical application where the frames come continuously, it requires a temporal segmentation of gestures, which in turn is a more challenging task. In our previous work [36], we proposed a method to deal with continuous recognition on other datasets using other methods of recognition. We may apply the same idea to this problem but with an adaptation for dealing with continuous gesture recognition.…”

Section: ) Discussionmentioning

confidence: 99%

Hand Gesture Recognition From Wrist-Worn Camera for Human–Machine Interaction

Nguyêñ

Tran³

et al. 2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

In this work, we study the ability to use hand gestures for human-machine interaction from wrist-worn sensors. Towards this goal, we design a wrist-worn prototype to capture RGB video stream of hand gestures. Then we built a new wrist-worn gesture dataset (named WiGes) with various subjects in interaction with home appliances in different environments. To the best of our knowledge, this is the first benchmark released for studying hand gestures from a wrist-worn camera. We then evaluate various CNN models for vision-based recognition. Furthermore, we deeply analyze the models that produce the best trade-off between accuracy, memory requirement, and computational cost. We point out that among studied architectures, MoviNet produces the highest accuracy. Then, we introduce a new MoviNet-based two-stream architecture that takes both RGB and optical flow into account. Our proposed architecture increases the Top-1 accuracy by 1.36% and 3.67% according to two evaluation protocols. Our dataset, baselines, and proposed model analysis give instructive recommendations for human-machine interaction using hand-held devices.INDEX TERMS Convolutional neural network, hand gesture recognition, human-machine interaction, wearable sensors.

show abstract

Section: ) Discussionmentioning

confidence: 99%