Combining HoloLens and Leap-Motion for Free Hand-Based 3D Interaction in MR Environments

Danglade, Florence; He, Junhui; Chardonnet, Jean-Rémy

doi:10.1007/978-3-030-58465-8_24

Cited by 8 publications

(8 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The virtual hand enables an isomorphic relationship between the actual and virtual hands [21]. Several works have utilized the virtual hand approach to interact with virtual elements [2], [22], [23]. The virtual pointer allows the selection of target items that are outside the user's reach with less actual hand movement.…”

Section: Related Workmentioning

confidence: 99%

Target selection method on the occluded and distant object in handheld augmented reality

Ismail

Halim²,

Talib³

et al. 2023

IJEECS

View full text Add to dashboard Cite

<span lang="EN-US">Existing interaction techniques within handheld augmented reality (AR) have frequently used touchscreen input (pure two-dimensional (2D) pointing and clicking) from the handheld device's display for target selection on the virtual object. However, performing accurate target selection on a distant target object becomes challenging as the target object will appear smaller when the distance increases. Aside from that, the difficulty increases in performing target selection when another virtual object obscures the distant virtual object. Therefore, this study aims to present a target selection method to perform the target selection. We enable the raycasting technique with real hand gesture for the target selection method on the occluded and distant object in handheld AR. The leap motion device is mounted at the back of the handheld device to track the real hand gesture. The markerless tracking technology of simultaneous localization and mapping (SLAM) is implemented to enable the AR environment. Based on the results, the aim of this study was achieved.</span>

show abstract

Section: Related Workmentioning

confidence: 99%

Target selection method on the occluded and distant object in handheld augmented reality

Ismail

Halim²,

Talib³

et al. 2023

IJEECS

View full text Add to dashboard Cite

show abstract

“…1. Grab and direct hand manipulation (Jailungka and Charoenseang, 2018;Kim et al, 2018;Frutos-Pascual et al, 2019;Ababsa et al, 2020;Bautista et al, 2020;Schäfer et al, 2022;Serrano et al, 2022;Su et al, 2022); 2. Manipulation with a virtual tool as a proxy (Jang et al, 2017); 3.…”

Section: Interactions Mainly Isomorphicmentioning

confidence: 99%

“…The user studies fromFrutos-Pascual et al (2019) show that Meta 2 interactions are preferred, qualified as more natural and useable, and require less cognitive charge according to the participants. Several other articles from the screening(Jailungka and Charoenseang, 2018;Kim et al, 2018;Ababsa et al, 2020;Bautista et al, 2020) also highlight the good usability of grasping objects. HoloLens 1 interactions only outperform Meta 2 interactions in terms of precision for the scaling which is a manipulation task.…”

mentioning

confidence: 99%

Hand interaction designs in mixed and augmented reality head mounted display: a scoping review and classification

Nguyen,

Gouin-Vallerand,

Amiri

2023

Front. Virtual Real.

View full text Add to dashboard Cite

Mixed reality has made its first step towards democratization in 2017 with the launch of a first generation of commercial devices. As a new medium, one of the challenges is to develop interactions using its endowed spatial awareness and body tracking. More specifically, at the crossroad between artificial intelligence and human-computer interaction, the goal is to go beyond the Window, Icon, Menu, Pointer (WIMP) paradigm humans are mainly using on desktop computer. Hand interactions either as a standalone modality or as a component of a multimodal modality are one of the most popular and supported techniques across mixed reality prototypes and commercial devices. In this context, this paper presents scoping literature review of hand interactions in mixed reality. The goal of this review is to identify the recent findings on hand interactions about their design and the place of artificial intelligence in their development and behavior. This review resulted in the highlight of the main interaction techniques and their technical requirements between 2017 and 2022 as well as the design of the Metaphor-behavior taxonomy to classify those interactions.

show abstract

“…Users could interact with digital applications using freehand gestures tracked by cameras on the wearable glasses. Numerous user scenarios and applications could be supported, such as manipulation and interaction with digital virtual objects [57][58][59] and annotation drawing for creating visual and spatial references [60].…”

Section: Freehand Gestural Interactionmentioning

confidence: 99%

Freehand Gestural Selection with Haptic Feedback in Wearable Optical See-Through Augmented Reality

et al. 2022

View full text Add to dashboard Cite

Augmented reality (AR) technologies can blend digital and physical space and serve a variety of applications intuitively and effectively. Specifically, wearable AR enabled by optical see-through (OST) AR head-mounted displays (HMDs) might provide users with a direct view of the physical environment containing digital objects. Besides, users could directly interact with three-dimensional (3D) digital artefacts using freehand gestures captured by OST HMD sensors. However, as an emerging user interaction paradigm, freehand interaction with OST AR still requires further investigation to improve user performance and satisfaction. Thus, we conducted two studies to investigate various freehand selection design aspects in OST AR, including target placement, size, distance, position, and haptic feedback on the hand and body. The user evaluation results indicated that 40 cm might be an appropriate target distance for freehand gestural selection. A large target size might lower the selection time and error rate, and a small target size could minimise selection effort. The targets positioned in the centre are the easiest to select, while those in the corners require extra time and effort. Furthermore, we discovered that haptic feedback on the body could lead to high user preference and satisfaction. Based on the research findings, we conclude with design recommendations for effective and comfortable freehand gestural interaction in OST AR.

show abstract

Combining HoloLens and Leap-Motion for Free Hand-Based 3D Interaction in MR Environments

Cited by 8 publications

References 10 publications

Target selection method on the occluded and distant object in handheld augmented reality

Target selection method on the occluded and distant object in handheld augmented reality

Hand interaction designs in mixed and augmented reality head mounted display: a scoping review and classification

Freehand Gestural Selection with Haptic Feedback in Wearable Optical See-Through Augmented Reality

Contact Info

Product

Resources

About