Investigating the Effect of Sensory Concurrency on Learning Haptic Spatiotemporal Signals

Abstract

A new generation of multimodal interfaces and interactions is emerging. Drawing on the principles of Sensory Substitution and Augmentation Devices (SSADs), these new interfaces offer the potential for rich, immersive human-computer interactions, but are difficult to design well, and take time to master, creating significant barriers towards wider adoption. Following a review of the literature surrounding existing SSADs, their metrics for success and their growing influence on interface design in Human Computer Interaction, we present a medium term (4-day) study comparing the effectiveness of various combinations of visual and haptic feedback (sensory concurrencies) in preparing users to perform a virtual maze navigation task using haptic feedback alone. Participants navigated 12 mazes in each of 3 separate sessions under a specific combination of visual and haptic feedback, before performing the same task using the haptic feedback alone. Visual sensory deprivation was shown to be inferior to visual & haptic concurrency in enabling haptic signal comprehension, while a new hybridized condition combining reduced visual feedback with the haptic signal was shown to be superior. Potential explanations for the effectiveness of the hybrid mechanism are explored, and the scope and implications of its generalization to new sensory interfaces is presented.

Stepping Into the Next Decade of Ubiquitous and Pervasive Computing: UbiComp and ISWC 2021

“…Many tactile and wearable interfaces face significant barriers due to the length of time necessary to master the technology and the difficulty that it takes to design them. Carson et al 86 conducted a literature review on existing sensory substitution and augmentation devices (SSAD) to examine their metrics for success and growing influence on interface design in order to widen their adoption in studies on human-computer interaction. SSADs can help create rich, immersive human-computer interactions, and hybrid visual and haptic feedback mechanisms have been shown to enable greater haptic signal comprehension.…”