Force and vibrotactile integration for 3D user interaction within virtual environments

Abstract

Proper integration of sensory cues facilitates 3D user interaction within virtual environments (VEs). Studies on multi-sensory integration of visual and haptic cues revealed that the integration follows maximum likelihood estimation (MLE). Little effort focuses however on integrating force and vibrotactile cues-two sub-categorical cues of the haptic modality. Hence, this paper presents an investigation on MLE's suitability for integrating these sub-categorical cues. Within a stereoscopic VE, human users performed a 3D interactive task of navigating a flying drone along a high-voltage transmission line in an inaccessible region and identifying defects on the line. The users had to identify defects via individual force or vibrotactile cues, and their combinations in colocated and dislocated settings. The co-located setting provided both cues on the right hand of the users; whereas the dislocated setting delivered the force and vibrotactile cues on the right hand and forearm of the users, respectively. Task performance of the users, such as completion time and accuracy, was assessed under each cue and setting. The presence of the vibrotactile cue promoted a better performance than the force cue alone. This was in agreement with the role of tactile cues in sensing surface properties, herein setting a baseline for using MLE. The task performance under the co-located setting indicated certain degrees of combining those under the individual cues. In contrast, the performance under the dislocated setting was alike that under the individual vibrotactile cue. These observations imply an inconclusiveness of MLE to integrate both cues in a co-located setting for 3D user interaction.

Vibrotactile and Force Collaboration within 3D Virtual Environments

“…Notwithstanding, what modifications of MLE are needed to remedy the two mismatches found in the existing work [11]? For handling the mean mismatch, we proposed mean-shifted MLE.…”

“…For this study, we developed a 3D stereoscopic VE for human participants to undertake an interactive task. The visual scene, interactive task and cues of the VE replicated their originals in the existing work [11]. Comparable hardware were used to deliver similar interactive experience to the participants who were biased to favor the vibrotactile cue over the force cue for the interactive task.…”

“…In contrast, this explanation is inconclusive when applying MLE to interpret empirical observations of collaboration between vibrotactile and force cues [11]. In the co-located setting of the cues, MLE elucidates the cue collaboration at certain degrees through a mean match between a prediction and an empirical observation.…”

“…Little work has been done to investigate these mismatches, hence we conducted an empirical study, in which each human participant undertook a 3D interactive task within a VE. We created the VE to replicate the visual scene, the task and the cues from those in the existing work [11]. Departing from a unbiased reliance on either of the cues in the existing work [11], each participant in the current study was instructed to favor the vibrotactile cue over the force cue.…”

“…We created the VE to replicate the visual scene, the task and the cues from those in the existing work [11]. Departing from a unbiased reliance on either of the cues in the existing work [11], each participant in the current study was instructed to favor the vibrotactile cue over the force cue. In other words, the participant had a biased reliance on the vibrotactile cue.…”