3d gesture recognition applying long short-term memory and contextual knowledge in a CAVE

Arsićc, Dejan; Roalter, Luis; Wöllmer, Martin; Eyben, Florian; Schuller, Björn W.; Kaiser, Moritz; Kranz, Matthias; Rigoll, Gerhard

doi:10.1145/1878039.1878047

Cited by 5 publications

(3 citation statements)

References 10 publications

(7 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In opposition to HWRA, handwriting recognition on surface using Deep Learning models like Bidirectional LSTMs [14], Connectionist Temporal Classifiers [15], and Multidimentional RNNs [16] have outperformed other baseline models (e.g., [17]). Similar Deep Learning models, such as Convolutional Neural Networks [18] and LSTMs [19], have also shown improved results in the domain of gesture recognition.…”

Section: Related Workmentioning

confidence: 99%

AirScript - Creating Documents in Air

Dash

Sahu

Shringi

et al. 2017

2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR)

View full text Add to dashboard Cite

This paper presents a novel approach, called AirScript, for creating, recognizing and visualizing documents in air. We present a novel algorithm, called 2-DifViz, that converts the hand movements in air (captured by a Myo-armband worn by a user) into a sequence of x, y coordinates on a 2D Cartesian plane, and visualizes them on a canvas. Existing sensor-based approaches either do not provide visual feedback or represent the recognized characters using prefixed templates. In contrast, AirScript stands out by giving freedom of movement to the user, as well as by providing a real-time visual feedback of the written characters, making the interaction natural. AirScript provides a recognition module to predict the content of the document created in air. To do so, we present a novel approach based on deep learning, which uses the sensor data and the visualizations created by 2-DifViz. The recognition module consists of a Convolutional Neural Network (CNN) and two Gated Recurrent Unit (GRU) Networks. The output from these three networks is fused to get the final prediction about the characters written in air. AirScript can be used in highly sophisticated environments like a smart classroom, a smart factory or a smart laboratory, where it would enable people to annotate pieces of texts wherever they want without any reference surface. We have evaluated AirScript against various well-known learning models (HMM, KNN, SVM, etc.) on the data of 12 participants. Evaluation results show that the recognition module of AirScript largely outperforms all of these models by achieving an accuracy of 91.7% in a person independent evaluation and a 96.7% accuracy in a person dependent evaluation.

show abstract

Section: Related Workmentioning

confidence: 99%

AirScript - Creating Documents in Air

Dash

Sahu

Shringi

et al. 2017

2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR)

View full text Add to dashboard Cite

show abstract

“…When this triangle is recognized by the infrared cameras, the position and orientation of the hand would be synchronized and transmitted to the master. Another infrared tracking system using active infrared cameras, is presented in [7]. The user's hand is equipped with a single marker.…”

Section: Related Workmentioning

confidence: 99%

“…Certainly, simple gestures acquired with a camera would provide a more natural user interface. Gesture recognition based on computer vision techniques for CAVE control has specific challenges due to low illumination intensity and space limitations to place devices inside the CAVE [7].…”

Section: Introductionmentioning

confidence: 99%

A System to Interact with CAVE Applications Using Hand Gesture Recognition from Depth Data

Leite

Duarte

Oliveira

et al. 2014

2014 XVI Symposium on Virtual and Augmented Reality

View full text Add to dashboard Cite

Human Computer Interaction (HCI) is a fundamental issue for virtual reality environments due to the need for natural approaches and comfortable devices. Such goals can be achieved using hand gestures to interact with the virtual reality engine. This paper presents a real-time system based on hand gesture recognition (HGR) for interaction with CAVE applications. The whole pipeline can be roughly divided into four steps: segmentation, feature extraction for bag-of-features construction, classification through multiclass support vector machine (SVM), generation of commands to control the application. We build a grammar based on the hand gesture classes to convert the classification results in control commands for an application running in a CAVE. The input is the depth stream data acquired from a Kinect device. The hand gesture recognition and command generation/execution approaches compose a clientserver plugin that is part of a CAVE system implemented based on the InstantReality architecture and the X3D standard. The results show that the implemented plugin is a promising solution. We achieve suitable recognition accuracy and efficient object manipulation in a virtual room representing a surgical environment visualized in the CAVE.

show abstract

Discussion

Schuller

2013

Intelligent Audio Analysis

View full text Add to dashboard Cite

3d gesture recognition applying long short-term memory and contextual knowledge in a CAVE

Cited by 5 publications

References 10 publications

AirScript - Creating Documents in Air

AirScript - Creating Documents in Air

A System to Interact with CAVE Applications Using Hand Gesture Recognition from Depth Data

Discussion

Contact Info

Product

Resources

About