Recognition of American Sign Language Gestures in a Virtual Reality Using Leap Motion

Vaitkevičius, Aurelijus; Taroza, Mantas; Blažauskas, Tomas; Damaševičius, Robertas; Maskeliūnas, Rytis; Woźniak, Marcin

doi:10.3390/app9030445

Cited by 85 publications

(42 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sideridis et al (2019) created a gesture recognition system for everyday gestures recorded with inertia measurement units, based on fast nearest neighbors, and support vector machine methods, whereas Yang and Sarkar (2006) chose to use an extension of HMMs. Vaitkevičius et al (2019) used also HMMs with the same purpose, gesture recognition, for the creation of virtual reality installations, as well as Williamson and Murray-Smith (2002), who used a combination of HMMs with a dynamic programming recognition algorithm, along with the granular synthesis method for gesture recognition with audio feedback. In a more industrial context, Yang et al (2007) used gesture spotting with HMMs to achieve efficient human-robot collaboration where realtime gesture recognition was performed with extended HMM methods like hierarchical HMMs (Li et al, 2017).…”

Section: Model-based Machine Learningmentioning

confidence: 99%

Human Movement Representation on Multivariate Time Series for Recognition of Professional Gestures and Forecasting Their Trajectories

et al. 2020

View full text Add to dashboard Cite

Human-centered artificial intelligence is increasingly deployed in professional workplaces in Industry 4.0 to address various challenges related to the collaboration between the operators and the machines, the augmentation of their capabilities, or the improvement of the quality of their work and life in general. Intelligent systems and autonomous machines need to continuously recognize and follow the professional actions and gestures of the operators in order to collaborate with them and anticipate their trajectories for avoiding potential collisions and accidents. Nevertheless, the recognition of patterns of professional gestures is a very challenging task for both research and the industry. There are various types of human movements that the intelligent systems need to perceive, for example, gestural commands to machines and professional actions with or without the use of tools. Moreover, the interclass and intraclass spatiotemporal variances together with the very limited access to annotated human motion data constitute a major research challenge. In this paper, we introduce the gesture operational model, which describes how gestures are performed based on assumptions that focus on the dynamic association of body entities, their synergies, and their serial and non-serial mediations, as well as their transitioning over time from one state to another. Then, the assumptions of the gesture operational model are translated into a simultaneous equation system for each body entity through state-space modeling. The coefficients of the equation are computed using the maximum likelihood estimation method. The simulation of the model generates a confidence-bounding box for every entity that describes the tolerance of its spatial variance over time. The contribution of our approach is demonstrated for both recognizing gestures and forecasting human motion trajectories. In recognition, it is combined with continuous hidden Markov models to boost the recognition accuracy when the likelihoods are not confident. In forecasting, a motion trajectory can be estimated by taking as minimum input two observations only. The performance of the algorithm has been evaluated using four industrial datasets that contain gestures and actions from a TV assembly line: the glassblowing industry, the gestural commands to automated guided vehicles as well as the human-robot collaboration in the automotive assembly lines. The hybrid approach State-Space and HMMs outperforms standard continuous HMMs and a 3DCNN-based end-to-end deep architecture.

show abstract

Section: Model-based Machine Learningmentioning

confidence: 99%

Human Movement Representation on Multivariate Time Series for Recognition of Professional Gestures and Forecasting Their Trajectories

et al. 2020

View full text Add to dashboard Cite

show abstract

“…It is expected that sensors for the acquisition of hands skeletal data will be improved soon. Therefore, work is underway to use them to recognize sign languages: American [15][16][17][18][19][20][21][22][23][24], Arabic [25][26][27][28], Australian [13], Indian [29][30][31], Mexican [32], Pakistani [33], and Polish [34].…”

Section: Related Workmentioning

confidence: 99%

“…Twenty-four static gestures of the American Finger Alphabet, shown ten times by twelve people, were recognized in [22]. Features based on the skeletal data returned by the LM controller were used.…”

Section: Related Workmentioning

confidence: 99%

Hand Posture Recognition Using Skeletal Data and Distance Descriptor

Kapuściński

Warchoł

2020

Applied Sciences

View full text Add to dashboard Cite

In this paper, a method for the recognition of static hand postures based on skeletal data was presented. A novel descriptor was proposed. It encodes information about distances between particular hand points. Five different classifiers were tested, including four common methods and a proposed modification of nearest neighbor classifier, which can distinguish between posture classes differing mostly in hand orientation. The experiments were performed using three challenging datasets of gestures from Polish and American Sign Languages. The proposed method was compared with other approaches found in the literature. It outperforms every compared method, including our previous work, in terms of recognition rate.

show abstract

“…Guillaume Plouffe et al [6] developed a natural gesture user interface that could track and recognize hand gesture based on depth data collected by a Kinect sensor. Aurelijus Vaitkeviźcius et al [14] presented a system that is capable of learning gestures by using the data from the Leap Motion device and the Hidden Markov classification algorithm. Although optical vision-based methods have good recognition performance, they are susceptible to illumination conditions and ambient infrared radiation [15].…”

Section: Introductionmentioning

confidence: 99%

A Method of Ultrasonic Finger Gesture Recognition Based on the Micro-Doppler Effect

Zeng

Kuang

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

With the popularity of small-screen smart mobile devices, gestures as a new type of human–computer interaction are highly demanded. Furthermore, finger gestures are more familiar to people in controlling devices. In this paper, a new method for recognizing finger gestures is proposed. Ultrasound was actively emitted to measure the micro-Doppler effect caused by finger motions and was obtained at high resolution. By micro-Doppler processing, micro-Doppler feature maps of finger gestures were generated. Since the feature map has a similar structure to the single channel color image, a recognition model based on a convolutional neural network was constructed for classification. The optimized recognition model achieved an average accuracy of 96.51% in the experiment.

show abstract

Recognition of American Sign Language Gestures in a Virtual Reality Using Leap Motion

Cited by 85 publications

References 47 publications

Human Movement Representation on Multivariate Time Series for Recognition of Professional Gestures and Forecasting Their Trajectories

Human Movement Representation on Multivariate Time Series for Recognition of Professional Gestures and Forecasting Their Trajectories

Hand Posture Recognition Using Skeletal Data and Distance Descriptor

A Method of Ultrasonic Finger Gesture Recognition Based on the Micro-Doppler Effect

Contact Info

Product

Resources

About