Visual display, pointing, and natural language

Angeli, Antonella De; Gerbino, Walter; Cassano, Giulia; Petrelli, Daniela

doi:10.1145/948496.948519

Cited by 19 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Smart Dictator feature aims to simulate (1) transcription of the end-user's speech into text, (2) parsing of natural speech editing requests from the end-user, and (3) editing of dictated text. Drawing inspiration from the key transcription functions of dictation applications such as Otter.ai 12 and video auto-caption generation system iTour 13 , the end-user interface provides a text editor that shows the dictated-text input and editing results in real-time (refer to Fig 2 .c).…”

Section: End-usermentioning

confidence: 99%

“…In particular, speech-based systems are expected to move towards greater complexity in response to the heightened demand for dynamic and open-ended interactions, potentially with non-voice complements [6,63]. This added demand for multi-modality makes the Wizard's task and the information bandwidth more complex [13,50], yet he/she cannot fumble because if they are too slow to respond, the end-user may react negatively, avoid using the new features being tested, or make mistakes that would not otherwise occur in a real system. Therefore, when the task is complex, ill-defined, multi-modal or multi-threaded, and/or dynamic, the burden on the Wizard can increase dramatically, to the point that they can become "lost in space," i.e., unable to determine where they need to go in the Wizarding interface [59,62].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Wizundry: A Cooperative Wizard of Oz Platform for Simulating Future Speech-based Interfaces with Multiple Wizards

Yen

et al. 2023

Proc. ACM Hum.-Comput. Interact.

View full text Add to dashboard Cite

Wizard of Oz (WoZ) as a prototyping method has been used to simulate intelligent user interfaces, particularly for speech-based systems. However, as our societies' expectations on artificial intelligence (AI) grows, the question remains whether a single Wizard is sufficient for it to simulate smarter systems and more complex interactions. Optimistic visions of 'what artificial intelligence (AI) can do' places demands on WoZ platforms to simulate smarter systems and more complex interactions. This raises the question of whether the typical approach of employing a single Wizard is sufficient. Moreover, while existing work has employed multiple Wizards in WoZ studies, a multi-Wizard approach has not been systematically studied in terms of feasibility, effectiveness, and challenges. We offer Wizundry, a real-time, web-based WoZ platform that allows multiple Wizards to collaboratively operate a speech-to-text based system remotely. We outline the design and technical specifications of our open-source platform, which we iterated over two design phases. We report on two studies in which participant-Wizards were tasked with negotiating how to cooperatively simulate an interface that can handle natural speech for dictation and text editing as well as other intelligent text processing tasks. We offer qualitative findings on the Multi-Wizard experience for Dyads and Triads of Wizards. Our findings reveal the promises and challenges of the multi-Wizard approach and open up new research questions.

show abstract

Section: End-usermentioning

confidence: 99%