Towards Taxonomy of Errors in Chat-oriented Dialogue Systems

Higashinaka, Ryuichiro; Funakoshi, Kotaro; Araki, Mikiya; Tsukahara, Hiroshi; Kobayashi, Yuka; Mizukami, Masahiro

doi:10.18653/v1/w15-4611

Cited by 63 publications

(54 citation statements)

References 20 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While overall miscommunication rate differed significantly among styles, it would be useful to understand whether different styles are associated with different types of miscommunication, as this may inform what error-handing algorithms the system should favor. Following Higashinaka et al (2015a), we categorize miscommunication in the corpus according to Response-level and Environmental-level ontologies. Though this ontology was designed to categorize errors made by a virtual agent dialogue system, a number of categories are applicable to communicative errors committed by human users in situated dialogues.…”

Section: Miscommunication Typesmentioning

confidence: 99%

Consequences and Factors of Stylistic Differences in Human-Robot Dialogue

Lukin

Pollard

Bonial

et al. 2018

Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue

View full text Add to dashboard Cite

This paper identifies stylistic differences in instruction-giving observed in a corpus of human-robot dialogue. Differences in verbosity and structure (i.e., single-intent vs. multi-intent instructions) arose naturally without restrictions or prior guidance on how users should speak with the robot. Different styles were found to produce different rates of miscommunication, and correlations were found between style differences and individual user variation, trust, and interaction experience with the robot. Understanding potential consequences and factors that influence style can inform design of dialogue systems that are robust to natural variation from human users.

show abstract

Section: Miscommunication Typesmentioning

confidence: 99%

Consequences and Factors of Stylistic Differences in Human-Robot Dialogue

Lukin

Pollard

Bonial

et al. 2018

Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue

View full text Add to dashboard Cite

show abstract

“…In the former approach, since the capability of the current chat-oriented dialogue systems is not good enough to always generate utterances that match the dialogue context (Higashinaka et al, 2015), engaging in chat for many turns might make the user's impression worse.…”

Section: Proposed Methodsmentioning

confidence: 99%

Small Talk Improves User Impressions of Interview Dialogue Systems

Kobori¹,

Nakano²,

Nagaoka³

2016

Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue

View full text Add to dashboard Cite

This paper addresses the problem of how to build interview systems that users are willing to use. Existing interview dialogue systems are mainly focused on obtaining information from users, thus they just repeatedly ask questions. We propose a method for improving user impressions by engaging in small talk during interviews. The system performs framebased dialogue management for interviewing and generates small talk utterances after the user answers the system's questions. Experimental results using a textbased interview dialogue system for diet recording showed the proposed method gives a better impression to users than interview dialogues without small talk. It is also found that generating too many small talk utterances makes user impressions worse because of the system's low capability of continuously generating appropriate small talk utterances.

show abstract

“…The dialogue format and rules were fully compliant with the Dialogue Breakdown Detection Challenge (DBDC) (see (Higashinaka et al, 2015a)). A dialogue is started by a system utterance, then user and the system communicate with one another.…”

Section: Dialogue Experimentsmentioning

confidence: 99%

“…In the field of text generation, there is a method that transforms individual characteristics in dialogue agent utterances using a method based on statistical machine translation (Mizukami et al, 2015). Other methods convert functional expressions into those that are suitable for a speaker's gender, age, and area of residence (Miyazaki et al, 2015) and closeness with a conversation partner (Miyazaki et al, 2016).…”

mentioning

confidence: 99%

Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue

Katagiri

Nakano

Fernández

et al. 2016

View full text Add to dashboard Cite

We extend special thanks to our Local co-Chairs, Ron Artstein and Alesia Gainer, and their team of student volunteers. We know SIGDIAL 2016 would not have been possible without Ron and Alesia, who invested so much effort in arranging the conference venue and accommodations, handling registration, making banquet arrangements, and handling numerous other preparations for the conference. The student volunteers for on-site assistance also deserve our appreciation.Ethan Selfridge, Sponsorships Chair, has earned our appreciation for recruiting and liaising with our conference sponsors, many of whom continue to contribute year after year. Sponsorships support valuable aspects of the program, such as the invited speakers and conference banquet. In recognition of this, we gratefully acknowledge the support of our sponsors: (Platinum level) Microsoft Research, Xerox and PARC, Intel, (Gold level) Facebook, (Silver level) Amazon Alexa, Interactions, Educational Testing Service, Honda Research Institute, and Yahoo!. At the same time, we thank Priscilla Rasmussen at the ACL for tirelessly handling the financial aspects of sponsorship for SIGDIAL 2016, and for securing our ISBN.iii We also thank the SIGdial board, especially officers Amanda Stent, Jason Williams and Kristiina Jokinen for their advice and support from beginning to end.Finally, we thank all the authors of the papers in this volume, and all the conference participants for making this stimulating event a valuable opportunity for growth in the research areas of discourse and dialogue. AbstractThis paper presents an end-to-end framework for task-oriented dialog systems using a variant of Deep Recurrent QNetworks (DRQN). The model is able to interface with a relational database and jointly learn policies for both language understanding and dialog strategy. Moreover, we propose a hybrid algorithm that combines the strength of reinforcement learning and supervised learning to achieve faster learning speed. We evaluated the proposed model on a 20 Question Game conversational game simulator. Results show that the proposed method outperforms the modular-based baseline and learns a distributed representation of the latent dialog state. IntroductionTask-oriented dialog systems have been an important branch of spoken dialog system (SDS) research (Raux et al., 2005; Young, 2006; Bohus and Rudnicky, 2003). The SDS agent has to achieve some predefined targets (e.g. booking a flight) through natural language interaction with the users. The typical structure of a task-oriented dialog system is outlined in Figure 1 (Young, 2006). This pipeline consists of several independently-developed modules: natural language understanding (the NLU) maps the user utterances to some semantic representation. This information is further processed by the dialog state tracker (DST), which accumulates the input of the turn along with the dialog history. The DST outputs the current dialog state and the dialog policy selects the next system action based on the dialog state. Then natural language gene...

show abstract

Towards Taxonomy of Errors in Chat-oriented Dialogue Systems

Cited by 63 publications

References 20 publications

Consequences and Factors of Stylistic Differences in Human-Robot Dialogue

Consequences and Factors of Stylistic Differences in Human-Robot Dialogue

Small Talk Improves User Impressions of Interview Dialogue Systems

Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue

Contact Info

Product

Resources

About