Zoom‐fwd: Efficient technique for 3D gestual interaction with distant and occluded objects in virtual reality

Abstract

In the last decade, there has been an extraordinary acceleration in immersive technologies, including virtual reality (VR) and augmented reality (AR). The VR interfaces have widely explored various 3D interaction‐based approaches. 3D interaction is one of the main features of any VR system. However, the problems while selecting and manipulating distant and occluded 3D virtual objects in VR are still unresolved and suffer from a lack of precision and accuracy. To interact with VR/AR systems, various devices such as VR headsets or gloves are used. These devices are not very reliable yet; in fact they are inconvenient and invasive to a person's comfort. Currently, the natural user interfaces (NUIs) are increasingly introduced in human computer interaction (HCI) systems as gestures recognition, which prove to be very useful in the improvement of the user engagement and presence sensing. More particularly, they provide more user‐friendly and nonintrusive 3D interaction methods and techniques. In this article, the Zoom‐fwd, which is an efficient 3D interaction technique is presented. The proposed technique uses gesture recognition for different 3D interaction tasks like selection and manipulation. This new approach allows an efficient interaction with distant and occluded objects, while providing a precise selection, even when the environment is crowded. The Zoom‐fwd technique is a software solution for a number of hardware and software problems. A user study is conducted to determine whether the proposed technique is more suitable when performing interaction tasks. The results show that the Zoom‐fwd technique provides effective interaction with distant and occluded objects, by improving the user task completion performance. In addition, this indicates that selection precision has been enhanced significantly with the Zoom‐fwd technique.

The role of digital interactive technology in cultural heritage learning: Evaluating a mid‐air gesture‐based interactive media of Ruihetu

“…The research provides evidence for the use of mid‐air gesture‐based digital media to promote learning about and cultural appreciation of historical artifacts. Our findings indicate that technological remediation can be a fruitful method of eliciting interest, engagement and presence sensing for cultural heritage 13,31 . Interactive media can attract users' attention and promote learning and memory in the context of informative teaching and learning, such as museums.…”

“…Our findings indicate that technological remediation can be a fruitful method of eliciting interest, engagement and presence sensing for cultural heritage. 13,31 Interactive media can attract users' attention and promote learning and memory in the context of informative teaching and learning, such as museums. However, care should be taken to set up scenarios, such as watching a video first to understand the background knowledge and then experiencing it, which is more in line with the laws of learning theory and can better improve memory.…”

Target selection method on the occluded and distant object in handheld augmented reality

“…A point of origin and a direction are needed to control the ray [26]. [12] Touch-based pointing with raycasting metaphor Handheld AR ✓ -2021 Li et al [9] Controller-based pointing with mirror metaphor VR ✓ ✓ 2021 Messaci et al [10] Gesture-based with zoom metaphor VR ✓ ✓ 2020 Qian et al [27] Gesture-based metaphor Handheld AR -✓ 2020 Yu et al [28] Controller-based pointing metaphors VR ✓ -2020 Sidenmark et al [29] Controller-based pointing with gaze-assisted metaphor VR ✓ -2019 Yin et al [30] Touch-based pointing metaphors Handheld AR ✓ -2018 Whitlock et al [8] Gesture-based metaphor AR ✓ -2017 Jung and Woo [13] Raycasting with target object duplication metaphor AR -✓ 2017 Bellarbi et al [14] Gesture-based pointing with zooming metaphor AR -✓ 2016 Yu and Kim [31] Finger-pointing metaphor AR -✓…”