Miscommunication handling in spoken dialog systems based on error-aware dialog state detection

Abstract

With the exponential growth in computing power and progress in speech recognition technology, spoken dialog systems (SDSs) with which a user interacts through natural speech has been widely used in human-computer interaction. However, error-prone automatic speech recognition (ASR) results usually lead to inappropriate semantic interpretation so that miscommunication happens easily. This paper presents an approach to error-aware dialog state (DS) detection for robust miscommunication handling in an SDS. Non-understanding (Non-U) and misunderstanding (Mis-U) are considered for miscommunication handling in this study. First, understanding evidence (UE), derived from the recognition confidence, is adopted for Non-U detection followed by Non-U recovery. For Mis-U with the recognized sentence containing uncertain recognized words, the partial sentences obtained by removing potentially misrecognized words from the input utterance are organized, based on regular expressions, as a tree structure to tolerate the deletion or rejection of keywords resulting from misrecognition for Mis-U DS modeling. Latent semantic analysis is then employed to consider the verified words and their n-grams for DS detection, including Mis-U and predefined Base DSs. Historical information-based n-grams are employed to find the most likely DS for the SDS. Several experiments were performed with a dialog corpus for the restaurant reservation task. The experimental results show that the proposed approach achieved a promising performance for Non-U recovery and Mis-U repair as well as a satisfactory task success rate for the dialogs using the proposed method.

Communication Adjustment in Engineering Professional and Student Project Meetings

“…While the literature identifies that communication styles and team roles change, it primarily addresses the human grounding processes [53,54] and the spoken dialogs system (interactions between human and computer) [55]. It is not specific about the reasons for such change, or the mechanisms.…”

Speech Recognition using Cross Correlation Algorithm Intended for Noise Reduction

“…Authors compared and summarized the wellknown methods used in various stages of speech recognition system. Wu et al [3] proposed an approach accomplished a promising execution for non-downplaying recuperation and misunderstanding repair and additionally an agreeable undertaking achievement rate for the exchanges utilizing the proposed technique. Chiang [4] discussed a parametric prosody coding approach for Mandarin discourse utilizing a various leveled prosodic model expressed that past research a novel parametric prosody coding approach for Mandarin discourse is proposed.…”

Follow-up Question Generation Using Pattern-based Seq2seq with a Small Corpus for Interview Coaching

“…In recent years, spoken dialogue systems (SDS) have been popular with the people who need some extra help, and have been extensively developed in a variety of areas, such as ticket booking, hotel reservations, interview coaching [1][2][3][4], etc. Among these applications, the interview coaching system attempts to simulate an interviewer to provide the mock interview to the users.…”