Multimodal feedback

Abstract

Multimodal interfaces have the potential to enhance a user's overall performance, especially when one perceptual channel, such as vision, is compromised. This research investigated how unimodal, b/modal, and trimodal feedback affected the performance of fully sighted users. Limited research exists that investigates how fully sighted users react to multimodal feedback forms, and to-date even less research is available that has investigated how users with visual impairments respond to multiple forms of feedback. A complex direct manipulation task, consisting of a series of search and selection drag-and-drop subtasks, was evaluated in this study. The multiple forms of feedback investigated were auditory, haptic and visual. Each form of feedback was tested alone and in combination. User performance was assessed through measures of workload and time. Workload was measured objectively and subjectively, through the physiological measure of pupil diameter and a portion of the NASA Task Load Index (TLX) workload survey, respectively. Time was captured by a measure of how long it took to complete a particular element of the task. The results demonstrate that multimodal feedback improves the performance of fully sighted users and offers great potential to users with visual impairments. As a result, this study serves as a baseline to drive the research and development of effective feedback combinations to enhance performance for individuals with visual impairments.

Assessing the Influence of Multimodal Feedback in Mobile-Based Musical Task Performance

“…It also provides a more flexible way for users to better perceive and avoid errors [9]. Research has shown that multimodal feedback positively impacts performance, perceived effort, error correction/avoidance, and error reaction time [10][11][12][13]. For example, Ref.…”

Assessing the Role of Gaze Tracking in Optimizing Humans-In-The-Loop Telerobotic Operation Using Multimodal Feedback

“…Multimodal feedback in teleoperation involves the use of more than one form of feedback during a teleoperation task. Vitense et al (2002) investigated how unimodal, bimodal, and trimodal feedback affect the complex direct manipulation performance of fully sighted users. Investigations were carried out on auditory feedback, haptic feedback, and visual feedback.…”