Development and validation of a Brazilian sign language database for human gesture recognition

Rezende, Tamires Martins; Almeida, Sílvia Grasiella Moreira; Guimarães, Frederico Gadelha

doi:10.1007/s00521-021-05802-4

Cited by 23 publications

(18 citation statements)

References 74 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The significance of multifaceted applications of hand gestures has increased in the industry and research fields [ 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 ]. Computer vision, pattern recognition, and human-computer interaction (HCI) are among the popular areas that involve 3D hand-gesture modeling [ 31 , 32 ].…”

Section: Methodsmentioning

confidence: 99%

Authorized Traffic Controller Hand Gesture Recognition for Situation-Aware Autonomous Driving

Mishra

Kim

Cha

et al. 2021

Sensors

View full text Add to dashboard Cite

An authorized traffic controller (ATC) has the highest priority for direct road traffic. In some irregular situations, the ATC supersedes other traffic control. Human drivers indigenously understand such situations and tend to follow the ATC; however, an autonomous vehicle (AV) can become confused in such circumstances. Therefore, autonomous driving (AD) crucially requires a human-level understanding of situation-aware traffic gesture recognition. In AVs, vision-based recognition is particularly desirable because of its suitability; however, such recognition systems have various bottlenecks, such as failing to recognize other humans on the road, identifying a variety of ATCs, and gloves in the hands of ATCs. We propose a situation-aware traffic control hand-gesture recognition system, which includes ATC detection and gesture recognition. Three-dimensional (3D) hand model-based gesture recognition is used to mitigate the problem associated with gloves. Our database contains separate training and test videos of approximately 60 min length, captured at a frame rate of 24 frames per second. It has 35,291 different frames that belong to traffic control hand gestures. Our approach correctly recognized traffic control hand gestures; therefore, the proposed system can be considered as an extension of the operational domain of the AV.

show abstract

Section: Methodsmentioning

confidence: 99%

Authorized Traffic Controller Hand Gesture Recognition for Situation-Aware Autonomous Driving

Mishra

Kim

Cha

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Rezende et al [8] também desenvolveu um conjunto de dados conhecido como MINDS-Libras. O conjunto consiste em 20 sinais repetidos 5 vezes por 12 interpretes, totalizando 1.200 sequências de dados gravados simultaneamente por um Microsoft Kinect v2 e uma DSLR da Canon.…”

Section: A Trabalhos Relacionadosunclassified

“…Os autores alegam que alguns vídeos foram perdidos e a versão final do conjunto de dados possui 1.155 sequências. O sistema de reconhecimento em [8] emprega Redes Convolucionais tridimensionais (3D-CNN) de forma similar à utilizada em Castro et al [9]. Os autores também avaliaram diferentes abordagens com imagens em nível de cinza, Histogramas de Gradientes Orientados (HOG), fluxo óptico, data augmentation e uso combinado das informações RGB e RGB-D. O melhor resultado foi obtido utilizando apenas os dados RGB convertidos em níveis de cinza com data augmentation, com uma acurácia média de 93,3%.…”

Section: A Trabalhos Relacionadosunclassified

Um sistema de reconhecimento de sinais em Libras usando CNN e LSTM

Stefano¹,

Passos²,

Gois³

et al. 2021

Anais Do XXXIX Simpósio Brasileiro De Telecomunicações E Processamento De Sinais

View full text Add to dashboard Cite

In Brazil, there are around 10 million hard of hearing and deaf people. However, the majority of Brazilians are not fluent in the Brazilian Sign Language (Libras). Many members of the hearing impaired and deaf community have communication issues in everyday life situations. Technological solutions can aid in mitigating this problem. This work proposes a semi-supervised method to identify and classify signals in Libras from Youtube videos. The routine starts by segmenting the videos through a measure of movement intensity. We catalog the video segments according to the corresponding subtitles, which we extract by employing Optical Character Recognition (OCR) if embedded. It composes an ad-hoc dataset that we use to train a Libras recognition system. A Convolutional Neural Network (CNN) performs feature extraction frame-by-frame, and a Recurrent Neural Network (RNN) models the time correlation between the features, thus classifying the signal. The proposed method can achieve accuracy up to 61.6% in the ad-hoc dataset used in this work.

show abstract

“…Existing sensors for a gesture recognition of movements of hands and arms include ultrasound [ 9 , 10 , 11 , 12 ], camera based vision [ 13 , 14 , 15 , 16 , 17 ], and radar [ 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 ]. Ultrasonic sensors have the advantage of relatively low price, but they have short detection distance.…”

Section: Introductionmentioning

confidence: 99%

“…Ultrasonic sensors have the advantage of relatively low price, but they have short detection distance. Camera based image sensors are a very common gesture recognition approach that use various popular CNN (Convolution Neural Network)-based deep learning models [ 13 , 14 , 15 , 16 , 17 ]. However, because camera based sensors are strongly dependent on surrounding environment factors like lighting, dust, and so on, it is necessary to design sensors very precisely for out-of-vehicle or outdoor use.…”

Section: Introductionmentioning

confidence: 99%

Foot Gesture Recognition Using High-Compression Radar Signature Image and Deep Learning

Song

Kim

et al. 2021

Sensors

View full text Add to dashboard Cite

Recently, Doppler radar-based foot gesture recognition has attracted attention as a hands-free tool. Doppler radar-based recognition for various foot gestures is still very challenging. So far, no studies have yet dealt deeply with recognition of various foot gestures based on Doppler radar and a deep learning model. In this paper, we propose a method of foot gesture recognition using a new high-compression radar signature image and deep learning. By means of a deep learning AlexNet model, a new high-compression radar signature is created by extracting dominant features via Singular Value Decomposition (SVD) processing; four different foot gestures including kicking, swinging, sliding, and tapping are recognized. Instead of using an original radar signature, the proposed method improves the memory efficiency required for deep learning training by using a high-compression radar signature. Original and reconstructed radar images with high compression values of 90%, 95%, and 99% were applied for the deep learning AlexNet model. As experimental results, movements of all four different foot gestures and of a rolling baseball were recognized with an accuracy of approximately 98.64%. In the future, due to the radar’s inherent robustness to the surrounding environment, this foot gesture recognition sensor using Doppler radar and deep learning will be widely useful in future automotive and smart home industry fields.

show abstract

Development and validation of a Brazilian sign language database for human gesture recognition

Cited by 23 publications

References 74 publications

Authorized Traffic Controller Hand Gesture Recognition for Situation-Aware Autonomous Driving

Authorized Traffic Controller Hand Gesture Recognition for Situation-Aware Autonomous Driving

Um sistema de reconhecimento de sinais em Libras usando CNN e LSTM

Foot Gesture Recognition Using High-Compression Radar Signature Image and Deep Learning

Contact Info

Product

Resources

About