Wizard of Oz Experimentation for Language Technology Applications: Challenges and Tools

Schlögl, Stephan; Doherty, Gavin; Luz, Saturnino

doi:10.1093/iwc/iwu016

Cited by 18 publications

(13 citation statements)

References 88 publications

(126 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…WoZ is a standard prototyping methodology in HCI that enables designers collect design requirements and evaluate features without actually developing the full system [47]. It also helps in uncovering di erent scenarios of use which might be expensive to study later on [101].…”

Section: Methodsmentioning

confidence: 99%

Feedpal

Yadav

Malik

Dabas

et al. 2019

Proc. ACM Hum.-Comput. Interact.

View full text Add to dashboard Cite

Use of chatbots in di erent spheres of life is continuously increasing since a couple of years. We attempt to understand the potential of chatbots for breastfeeding education by conducting an Wizard-of-Oz experiment with 22 participants. Our participants included breastfeeding mothers and community health workers from the slum areas of Delhi, India. We prototyped our chatbot as an interactive question-answering application and analyzed users' interaction patterns, perceptions, and contexts of use. The chatbot use cases emerged primarily as the rst line of support. The participants, especially the mothers, were enthusiastic with the opportunity to ask questions and get reliable answers. We also observed the in uencing role of female relative, e.g. mothers-in-law, in breastfeeding practices. Our analysis of user information-seeking suggests that a majority of questions (88%) are of nature that can be answered by a chatbot application. We further observe that the queries are embedded deeply into myths and existing belief systems. Therefore requiring the designers to focus on subtle aspects for providing information such as positive reinforcement and contextual sensitivity. Further, we discuss, di erent societal and ethical issues associated with Chatbot usage for a public health topic such as breastfeeding education.

show abstract

Section: Methodsmentioning

confidence: 99%

Feedpal

Yadav

Malik

Dabas

et al. 2019

Proc. ACM Hum.-Comput. Interact.

View full text Add to dashboard Cite

show abstract

“…use case descriptions, sketches, scenarios, etc.). One of the used methods, which is particularly popular when building technology based on natural language (Schlögl et al 2015) or other types of artificial intelligence driven applications (Dahlbäck et al 1993), was Wizard of Oz (WOZ). The key principle of the WOZ method is that study participants believe they are interacting with an autonomous system while actually the system's actions are controlled by a human (i.e.…”

Section: Building the Virtual Coach Interacting Environmentmentioning

confidence: 99%

Analysis of the interaction between elderly people and a simulated virtual coach

Justo

Zouari

Palmero

et al. 2020

J Ambient Intell Human Comput

Self Cite

View full text Add to dashboard Cite

The EMPATHIC project develops and validates new interaction paradigms for personalized virtual coaches (VC) to promote healthy and independent aging. To this end, the work presented in this paper is aimed to analyze the interaction between the EMPATHIC-VC and the users. One of the goals of the project is to ensure an end-user driven design, involving senior users from the beginning and during each phase of the project. Thus, the paper focuses on some sessions where the seniors carried out interactions with a Wizard of Oz driven, simulated system. A coaching strategy based on the GROW model was used throughout these sessions so as to guide interactions and engage the elderly with the goals of the project. In this interaction framework, both the human and the system behavior were analyzed. The way the wizard implements the GROW coaching strategy is a key aspect of the system behavior during the interaction. The language used by the virtual agent as well as his or her physical aspect are also important cues that were analyzed. Regarding the user behavior, the vocal communication provides information about the speaker’s emotional status, that is closely related to human behavior and which can be extracted from the speech and language analysis. In the same way, the analysis of the facial expression, gazes and gestures can provide information on the non verbal human communication even when the user is not talking. In addition, in order to engage senior users, their preferences and likes had to be considered. To this end, the effect of the VC on the users was gathered by means of direct questionnaires. These analyses have shown a positive and calm behavior of users when interacting with the simulated virtual coach as well as some difficulties of the system to develop the proposed coaching strategy.

show abstract

“…We used the WebWOZ prototyping platform 6 , a tool specifically geared towards designing and evaluating natural language based interactions [16]. It is entirely web-based and allows for the integration of existing language technologies such as automatic speech recognition and text-to-speech synthesis modules.…”

Section: The Woz Prototyping Toolmentioning

confidence: 99%

LifeLine Dialogues with Roberta

López¹,

Ratni²,

Trọng³

et al. 2017

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

This paper describes work on dialogue data collection and dialogue system design for personal assistant humanoid robots undertaken at eNTERFACE 2016. The emphasis has been on the system's speech capabilities and dialogue modeling of what we call LifeLine Dialogues, i.e. dialogues that help people tell stories about their lives. The main goal behind this type of application is to help elderly people exercise their speech and memory capabilities. The system further aims at acquiring a good level of knowledge about the person's interests and thus is expected to feature open-domain conversations, presenting useful and interesting information to the user. The novel contributions of this work are: (1) a flexible spoken dialogue system that extends the Ravenclawtype agent-based dialogue management model with topic management and multi-modal capabilities, especially with face recognition technologies, (2) a collection of WOZ-data related to initial encounters and presentation of information to the user, and (3) the establishment of a closer conversational relationship with the user by utilizing additional data (e.g. context, dialogue history, emotions, user goals, etc.).

show abstract

Wizard of Oz Experimentation for Language Technology Applications: Challenges and Tools

Cited by 18 publications

References 88 publications

Feedpal

Feedpal

Analysis of the interaction between elderly people and a simulated virtual coach

LifeLine Dialogues with Roberta

Contact Info

Product

Resources

About