Haptic feedback combined with movement sonification using a friction sound improves task performance in a virtual throwing task

Abstract

In this paper we present a study on the effects of auditory-and haptic feedback in a virtual throwing task performed with a pointbased haptic device. The main research objective was to investigate if and how task performance and perceived intuitiveness is affected when interactive sonification and/or haptic feedback is used to provide real-time feedback about a movement performed in a 3D virtual environment. Emphasis was put on task solving efficiency and subjective accounts of participants' experiences of the multimodal interaction in different conditions. The experiment used a within-subjects design in which the participants solved the same task in different conditions: visual-only, visuohaptic, audiovisual and audiovisuohaptic. Two different sound models were implemented and compared. Significantly lower error rates were obtained in the audiovisuohaptic condition involving movement sonification based on a physical model of friction, compared to the visual-only condition. Moreover, a significant increase in perceived intuitiveness was observed for most conditions involving haptic and/or auditory feedback, compared to the visual-only condition. The main finding of this study is that multimodal feedback can not only improve perceived intuitiveness of an interface but that certain combinations of haptic feedback and movement sonification can also contribute with performance-enhancing properties. This highlights the importance of carefully designing feedback combinations for interactive applications.

Study of Multimodal Interfaces and the Improvements on Teleoperation

“…Thus full conclusions cannot be drawn with our static scenario and further investigation is required to confirm. Our findings show that haptic feedback leads to better performance which is supported by some studies [37], but contradicts others [45]. The latter study found no significant effect of haptic feedback in a virtual throwing task.…”

“…On one hand, an audiovisuohaptic interface did not significantly increase performance during a teleoperated navigation task [9], but operator's spatial ability and subjective performance did increase compared to using fewer interfaces. In another study, an audiovisuohaptic interface was implemented to assess performance in visual throwing tasks [45]. While not exhausting all comparisons of the interface or implementing varying task complexities, their results show that point-based haptic devices and moreover auditory feedback did not significantly contribute to the improvement of task performance.…”

Forehead Tactile Hallucination Is Augmented by the Perceived Risk and Accompanies Increase of Forehead Tactile Sensitivity

“…We interpret this result as the nervous system providing a warning signal when faced with a threat. As tactile feedback is a more intuitive sensation than visual feedback, with less processing needs and processing delay [33], the nervous system may activate tactile feedback for better preparation of the body to the potential threat. The prior studies on phantom limb pain for amputee patients also suggest that the nervous system provides a warning signal to not repeat the mistake to lose the limb [23].…”

Exploring crossmodal perceptual enhancement and integration in a sequence-reproducing task with cognitive priming

“…graded crossmodal stimuli, the implementation of CCs in real-world scenarios remains limited. After all, a variety of crossmodal interaction cases involve consistent perception-action loops, either for spatial localization [19], graph sonification [3], or motor skill learning [17].…”