Simulation-based dialogue design for speech-controlled telephone services

Abstract

A design methodology for speech-controlled telephone services has been developed using W~ard-of-Oz simulations as the principal mechanism for evaluating and getting input for dialogue design. This methodology may enable service developers to support dialogues that are optimal with respect to naturalness, especially on a pragmatic level, given the technical restrictions at hand. METHODOLOGY DEVELOPMENTAutomatic speech understanding is becoming an increasingly attractive option for providing advanced services to a broad audience while exploiting optimally the limited bandwidth of the telephone channel. Most existing speeeh-controlled services are based around small voeabuhries and isolated word recognition, but as continuous speech recognition technology matures, this will change. Our hypothesis in the DISA (Design for Input Speech Adaptation) project has been that, regardless of the quality of the speech recognition and natural language processing technology available, all such services may benefit from having dialogues derived from task analysis and Wizard-of-Oz simulation studies. In other words, even though natural syntax and semantics cannot be supported, natural pragmatic may still be of use (and to some extent compensate for the limitations).While our studies have produced both serviee-and dialogue-specific da~obtaining meta-results has been the real objective, giving us feedback on how to pursue simulationcentered dialogue design.Permission to copy without fee all or part of this material k granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of ACM. To copy otherwise, or to republish, requires a fee and/or specific permission. CHI' Companion 95, Denver, Colorado, USA 0 1995 ACM 0-89791 -755 -3/95/0005 ...$3.50 WIZARD-OF-OZ SIMULATIONS Several studies have been undertaken in the area of simulating speech-understanding systems [1,3,4], as well as systems interacting through written natural language [2,5], giving suggestions on how to set up such experiments. In the same spirit as most of these experiments, DISA used the simulation-set-up shown in figure 1.The DISA dialogue design methodology comprises at least two separate stages of simulation. The tirst stage is based on soliciting unrestricted, task-oriente~human-human dialogues. For the second round of simulations, a rudimentary dialogue model is used, describing how the target service should behave in different stages of the dialogue. RESULTS Methodological resultsThrough basing the simulations in the second stage on a rudimentary dialogue model, the behaviour of an automated service could be approximated. A support tool, the Wizard's Answering Device (WAND) represented this basic organization of the different parts of the dialogues as a set of panels each having several groups of messages, arranged according to subtask, giving the wizard guidance on what answers were appropriate in a gi...

Look at Me When I Talk to You: A Video Dataset to Enable Voice Assistants to Recognize Errors

“…Many are voice-enabled assistants, such as Microsoft's Cortana [46], Amazon's Alexa [1], Apple's Siri [5], or Google Assistant [30]. In some of the first-generation voice assistant research published in the larger HCI community, such as [13,51,65], the computer-generated voice that people were speaking with was often on the other side of a phone line. In other cases, like the embodied conversational agents of Cassell, Sullivan, Churchill and Prevost [14], the voice was front-ended by on-screen virtual agents.…”

Approaches to gathering realistic training data for speech translation systems

“…One could imagine that genuine humanhuman conversations would provide the best yardstick for linguistic training data, but this is not true given that the linguistic . performance models for users engaging in dialogue with a machine varies with the behaviour of the system [5]. There is also the ethical issue of ''bugging'' people's conversations, which is most critical if the speakers could be identified and i f the dialogue contains sensitive or personal information.…”