Contextual Dialogue Act Classification for Open-Domain Conversational Agents

Abstract

Identifying the topic (domain) of each user's utterance in opendomain conversational systems is a crucial step for all subsequent language understanding and response tasks. In particular, for complex domains, an utterance is often routed to a single component responsible for that domain. Thus, correctly mapping a user utterance to the right domain is critical. To address this problem, we introduce ConCET: a Concurrent Entity-aware conversational Topic classifier, which incorporates entity-type information together with the utterance content features. Specifically, ConCET utilizes entity information to enrich the utterance representation, combining character, word, and entity-type embeddings into a single representation. However, for rich domains with millions of available entities, unrealistic amounts of labeled training data would be required. To complement our model, we propose a simple and effective method for generating synthetic training data, to augment the typically limited amounts of labeled training data, using commonly available knowledge bases as to generate additional labeled utterances. We extensively evaluate ConCET and our proposed training method first on an openly available human-human conversational dataset called Self-Dialogue, to calibrate our approach against previous state-of-the-art methods; second, we evaluate ConCET on a large dataset of human-machine conversations with real users, collected as part of the Amazon Alexa Prize. Our results show that ConCET significantly improves topic classification performance on both datasets, including 8-10% improvements over state-of-the-art deep learning methods. We complement our quantitative results with detailed analysis of system performance, which could be used for further improvements of conversational agents.

User Intent Inference for Web Search and Conversational Agents

“…First, for user intent classification, I proposed a novel method, CDAC (Contextual Dialogue Act Classifier), a simple yet effective deep learning approach for contextual dialogue act (broad intent) classification. Specifically, we used transfer learning to adapt models trained on human-human conversations to predict dialogue acts in human-machine dialogues [2]. Then, for topic classification, we introduced a Concurrent Conversational Entity-aware Topic classifier (ConCET), which incorporates entity-type information together with the utterance content features.…”

“…Then, for topic classification, we introduced a Concurrent Conversational Entity-aware Topic classifier (ConCET), which incorporates entity-type information together with the utterance content features. Specifically, ConCET utilizes entity information to enrich the utterance representation, combining character, word, and entity-type embeddings into a single representation [3]. Finally, we proposed a Conversational Satisfaction prediction model specifically designed for open-domain conversational agents, called Con-vSAT.…”

A Conceptual Data Modelling Framework for Context-Aware Text Classification

“…It does not consider polysemy of words and also manually detect topics which are time consuming. Topic identification in dialog systems is an interesting application of Topic Modeling [2]. It is applied in human and Chabot systems.…”

Proceedings of the 1st Workshop on NLP for Positive Impact

“…Once labeled conversational data is available, it can be used to create generative statistical systems that take a sentence and a prior Dialogue Act as input and provide the next most like Dialogue Act for the conversation. Prior research has used this information to analyze both human-human conversations and better facilitate human-machine conversations (Ahmadvand et al, 2019). However, research in Dialogue Act classification has not included conversations with individuals who do not rely solely on verbal speech to communicate.…”