A Flexible Dual Task Paradigm for Evaluating an Embodied Conversational Agent: Modality Effects and Reaction Time as an Index of Cognitive Load

Abstract

A new experimental method based on the dual task paradigm is used to evaluate speech intelligibility of an embodied conversational agent (ECA). The experiment consists of the manipulation of auditory-visual (AV) versus auditory-only (A) presentation of speech. In the dual task, participants perform two tasks concurrently. The secondary task is sensitive to cognitive processing demands of the primary task. In the primary task participants either shadowed words or named the superordinate categories to which words belonged, as the word items were spoken by the ECA under A or AV conditions. Reaction time (RT) on the secondary task-swatting a fly on the ECA face-was affected by the difficulty of the concurrent task. The secondary RT was affected by modality of presentation of the primary task. Using a relatively primitive ECA, RT on the secondary task was significantly slower when shadowing occurred in AV versus A conditions. The benefits of this evaluation system, that returns quantitative behavioural data and self-report ratings, are discussed.

Embodiment

“…We know that participant processing of auditory visual as oppose to auditory only information, increases cognitive load and this has been shown to reflect in increased participant reaction times [30]. So, what is clear is that the audience needs to be able to make a tangible visual connection with the projections for a sufficiently long duration of time, in order to view, read, digest and identify connections between the displayed words, input text, ECA output and their own thought processes.…”