The impact of unwanted multimodal notifications

Abstract

Multimodal interaction can be used to make home care technology more effective and appropriate, particularly for people with sensory impairments. Previous work has revealed how disruptive notifications in different modalities are to a home-based task, but has not investigated how disruptive unwanted notifications might be. An experiment was conducted which evaluated the disruptive effects of unwanted notifications when delivered in textual, pictographic, abstract visual, speech, earcon, auditory icon, tactile and olfactory modalities. It was found that for all the modalities tested, both wanted and unwanted notifications produced similar reductions in error rate and task success, independent of modality. The results demonstrate the need to control and limit the number of unwanted notifications delivered in the home and contribute to a large body of work advocating the inclusion of multiple interaction modalities.

Multiple notification modalities and older users

“…This was confirmed by Warnock et al [10] who evaluated text, pictogram, abstract visual, speech, Earcon, Auditory Icon, tactile and olfactory modalities. Warnock et al [9] also found that target notifications (which required a response) and distractor notifications (which should be ignored) would induce the same negative effects (reduced success rate and pause in activity) regardless of the delivery modality. That work was carried out with younger users aged 18-30, but was extended to include older participants aged 50-90 for a direct comparison of modal notification performance between age groups [11].…”

“…Many researchers have called for systems that offer multiple modalities and are able to switch between them to suit the user, environment or message being delivered [2,6,7,9,12]. Research has shown that adaptable technology such as this can deliver notifications in a manner that is more acceptable Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.…”

“…Warnock et al [9,10] argued that "there is no 'best modality'; there is a best modality for the situation", and that the most appropriate modality should be selected based on a number of factors such as message urgency, user impairment and social context. Vastenburg et al [8] found that delivering low/high urgency information using less/more salient methods increased user acceptability.…”

The design and field observation of a haptic notification system for timing awareness during oral presentations

“…Performance degradation is seen in many studies of how sensory and attentional resources interact and compete in a wide variety of task overload situations involving computation, communication, collaboration, and recovery from disruption [3,8,47]. Monitoring time while speaking publicly is no exception: time estimation accuracy suffers under cognitive load [39], and self-monitoring can increase self-interruption [18].…”

“…Maintaining time awareness is one more drain. Supporting this need under a different channel might free some audition or vision and be less distracting due to lower mental demands [13,47], while adding an information stream (time awareness) that was unavailable. Automated (pre-set) cues in any modality could theoretically privately reduce the need to poll (repeatedly check) timing parameters (e.g., auditory nonverbal clicks through an earbud device, flashing lights on the podium), but studies show that presenting information haptically reduces cognitive load on vision and audition, optimizing attention and sensory resources [20,25,28].…”