Review of Three-Dimensional Human-Computer Interaction with Focus on the Leap Motion Controller

Abstract

Modern hardware and software development has led to an evolution of user interfaces from command-line to natural user interfaces for virtual immersive environments. Gestures imitating real-world interaction tasks increasingly replace classical two-dimensional interfaces based on Windows/Icons/Menus/Pointers (WIMP) or touch metaphors. Thus, the purpose of this paper is to survey the state-of-the-art Human-Computer Interaction (HCI) techniques with a focus on the special field of three-dimensional interaction. This includes an overview of currently available interaction devices, their applications of usage and underlying methods for gesture design and recognition. Focus is on interfaces based on the Leap Motion Controller (LMC) and corresponding methods of gesture design and recognition. Further, a review of evaluation methods for the proposed natural user interfaces is given.

Open data exploration in virtual reality: a comparative study of input technology

“…Their results report on the users ability to retain more structural information and spatial understanding about the network compared to the 2D screen setup (Betella et al 2014). Bachmann et al (2018) surveyed HCI techniques with a focus on 3D gestural input using the Leap Motion controller, providing a good overview of relevant methods and evaluation techniques. Gusai et al (2017) conducted a study comparing gestural input (using a Leap Motion controller) and controller-based (HTC Vive) interaction in an collaborative scenario.…”

“…The NASA Task Load Index (TLX) is a method to let a user estimate the experienced workload when operating an interactive system (Hart and Staveland 1988;Hart 2006), and has been applied for VR and input technology scenarios in different contexts before (Lackey et al 2016;Bachmann et al 2018). The TLX investigates six factors, namely physical demand, mental demand, temporal demand, effort, frustration, and the user's perceived own performance.…”

Fusing Visual Attention CNN and Bag of Visual Words for Cross-Corpus Speech Emotion Recognition

“…Emotion recognition (ER) plays an important role in human-computer interaction (HCI) [ 1 ]. During the last few years, numerous approaches have been proposed using different modalities (e.g., speech, facial expressions, and gestures) [ 2 , 3 , 4 , 5 ].…”

Recognition of American Sign Language Gestures in a Virtual Reality Using Leap Motion

“…Hand gesture recognition is widely researched as it can be applied to different areas such as human-computer interaction [1], robotics [2], computer games [3], education [4], automatic sign-language interpretation [5], decision support for medical diagnosis of motor skills disorders [6], recognition of children with autism [7], home-based rehabilitation [8,9], virtual training [10] and virtual surgery [11]. In industry, gesture recognition can be used in areas requiring very high precision such as to control devices such as robot hands [12] or industrial equipment.…”