Hand Gesture Recognition in Video Sequences Using Deep Convolutional and Recurrent Neural Networks

Obaid, Falah; Babadi, Amin; Yoosofan, Ahmad

doi:10.2478/acss-2020-0007

Cited by 17 publications

(7 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Their model when used on the VIVA challenge dataset, attained classification precision of 77.5%. In [17], the researchers introduced two models for hand gesture recognition. The first model consists of CNN and an RNN-LSTM network.…”

Section: Related Workmentioning

confidence: 99%

Dynamic Hand Gesture Recognition Using 3D-CNN and LSTM Networks

Rehman¹,

Ahmed²,

Khan³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Recognition of dynamic hand gestures in real-time is a difficult task because the system can never know when or from where the gesture starts and ends in a video stream. Many researchers have been working on visionbased gesture recognition due to its various applications. This paper proposes a deep learning architecture based on the combination of a 3D Convolutional Neural Network (3D-CNN) and a Long Short-Term Memory (LSTM) network. The proposed architecture extracts spatial-temporal information from video sequences input while avoiding extensive computation. The 3D-CNN is used for the extraction of spectral and spatial features which are then given to the LSTM network through which classification is carried out. The proposed model is a light-weight architecture with only 3.7 million training parameters. The model has been evaluated on 15 classes from the 20BN-jester dataset available publicly. The model was trained on 2000 video-clips per class which were separated into 80% training and 20% validation sets. An accuracy of 99% and 97% was achieved on training and testing data, respectively. We further show that the combination of 3D-CNN with LSTM gives superior results as compared to MobileNetv2 + LSTM.

show abstract

Section: Related Workmentioning

confidence: 99%

Dynamic Hand Gesture Recognition Using 3D-CNN and LSTM Networks

Rehman¹,

Ahmed²,

Khan³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

show abstract

“…They attained an overall accuracy of 85.46 %. Fahad Obaid et al [71] proposed a model comprised of two stages to solve the hand gesture recognition (HGR) problem in Video Sequences. The first stage is preprocessing, while the second is to classify, label the frames and recognize the hand gestures through deep learning.…”

Section: Related Workmentioning

confidence: 99%

“…To shorten the training period, we began reducing the numbers of extracted frames (1-10) and discovered that 5 frames from each video sequence at equal intervals were sufficient to reflect the dynamics of the gestures without compromising the prediction accuracy. Furthermore, previous researches demonstrate that human action recognition may be accomplished by just a few frames (1-7 frames) [77,78].…”

Section: B Pre-processingmentioning

confidence: 99%

Helping Hearing-Impaired in Emergency Situations: A Deep Learning-Based Approach

et al. 2022

View full text Add to dashboard Cite

Hearing-impaired people use sign language to express their thoughts and emotions and reinforce information delivered in daily conversations. Though they make a significant percentage of any population, the majority of people can't interact with them due to limited or no knowledge of sign languages. Sign language recognition aims to detect the significant motions of the human body, especially hands, analyze them and understand them. Such systems may become life-saving when hearing-challenged people are in desperate situations like heart attacks, accidents, etc. In the present study, deep learning-based hand gesture recognition models are developed to accurately predict the emergency signs of Indian Sign Language (ISL). The dataset used contains the videos for eight different emergency situations. Several frames were extracted from the videos and are fed to three different models. Two models are designed for classification, while one is an object detection model, applied after annotating the frames. The first model consists of a three-dimensional convolutional neural network (3D CNN), while the second comprises of a pre-trained VGG-16 and a recurrent neural network with a long short-term memory (RNN-LSTM) scheme. The last model is based on YOLO (You Only Look Once) v5, an advanced object detection algorithm. The prediction accuracies of the classification models were 82% and 98%, respectively. YOLO based model outperformed the rest and achieved an impressive mean average precision of 99.6%.

show abstract

“…Jarrín en 2020 [10], usaron una red neuronal convolucional pre entrenada que ha sido entrenada en aproximadamente 1.2 millones de imágenes y el Algoritmo de Estimación de Pose Humana (OpenPose), propone un sistema de detección del ángulo articular en los movimientos de miembro superior para evaluación en fisioterapia mediante visión artificial, éste logró un nivel de confiabilidad del 92,60%, implementado con python y tensorflow. Obaid et al en 2020 [11], utilizaron redes neuronales convolucionales profundas (CNN) y recurrentes (LSTM) para el reconocimiento de gestos con las manos en secuencias de video, en su trabajo crearon dos modelos para determinar la arquitectura óptima de CNN y LSTM. El modelo uno obtuvo un 82% de precisión trabajando con el canal de color y un 89% con el canal de profundidad, sin embargo, el segundo modelo logró 93% de precisión trabajando con el canal de color y profundidad.…”

Section: Trabajos Relacionadosunclassified

Prototipo para el reconocimiento de video en tiempo real de los ejercicios del método Klapp usando redes neuronales recurrentes

Rojas¹,

Jiménez²,

Téllez³

et al. 2022

progmat

View full text Add to dashboard Cite

En la actualidad la telerehabilitación ha tomado un gran auge debido a los acontecimientos de la COVID-19, éste trata de la aplicación de un programa bajo la supervisión remota, usando las tecnologías de información y comunicación, eliminando limitantes de las personas para su desplazamiento, minimizando riesgos y aportando mayor cumplimiento a los programas de rehabilitación y reacondicionamiento. El presente trabajo propone el desarrollo de una herramienta para el reconocimiento en tiempo real de ejercicios del método fisioterapéutico Klapp, basándonos en el concepto de telerehabilitación. El objetivo de este proyecto es dar apoyo a los fisioterapeutas para dar seguimiento al tratamiento de rehabilitación o reacondicionamiento. La herramienta permite evaluar la ejecución en tiempo real de cada uno de los ejercicios para informarle al paciente si lo realizó correctamente, y el fisioterapeuta podrá consultar dichos resultados. El desarrollo del proyecto fue gestionado bajo el modelo V como referencia para cada una de las actividades realizadas. La técnica utilizada para el reconocimiento son las redes neuronales recurrentes, pues su arquitectura permite modelar la siguiente predicción basándose en la información pasada y lograr modelar datos en secuencia como videos. Y para la implementación fueron empleadas las herramientas Keras, TensorFlow, Python, librerías de OpenCV y OpenPose, con el cual se obtuvieron los puntos de referencias de los ejercicios basándonos en los ángulos de Cobb. Los resultados obtenidos de reconocimiento de la red recurrente fueron medidos bajo la métrica de exactitud (Accuracy), logrando un 97% considerando 6 ejercicios, los cuales comprenden los ejercicios del método Klapp.

show abstract

Hand Gesture Recognition in Video Sequences Using Deep Convolutional and Recurrent Neural Networks

Cited by 17 publications

References 17 publications

Dynamic Hand Gesture Recognition Using 3D-CNN and LSTM Networks

Dynamic Hand Gesture Recognition Using 3D-CNN and LSTM Networks

Helping Hearing-Impaired in Emergency Situations: A Deep Learning-Based Approach

Prototipo para el reconocimiento de video en tiempo real de los ejercicios del método Klapp usando redes neuronales recurrentes

Contact Info

Product

Resources

About