Contextual Topic Modeling For Dialog Systems

Abstract

Accurate prediction of conversation topics can be a valuable signal for creating coherent and engaging dialog systems. In this work, we focus on context-aware topic classification methods for identifying topics in free-form human-chatbot dialogs. We extend previous work on neural topic classification and unsupervised topic keyword detection by incorporating conversational context and dialog act features. On annotated data, we show that incorporating context and dialog acts leads to relative gains in topic classification accuracy by 35% and on unsupervised keyword detection recall by 11% for conversational interactions where topics frequently span multiple utterances. We show that topical metrics such as topical depth is highly correlated with dialog evaluation metrics such as coherence and engagement implying that conversational topic models can predict user satisfaction. Our work for detecting conversation topics and keywords can be used to guide chatbots towards coherent dialog.

MIDAS: A Dialog Act Annotation Scheme for Open Domain HumanMachine Spoken Conversations

“…In this paper, we design a dialog annotation scheme specifically for humanmachine social chitchat conversations without any topic constraints. Khatri et al (2018) introduces a human-machine dialog act annotation scheme with 14 tags. However, the scheme is designed for modeling conversation topics instead of training dialog act predictors.…”

“…Contextual information plays an important role in dialog act prediction (Liu et al, 2017;Khatri et al, 2018). We consider two methods to represent previous turns: the actual utterance (text), and the dialog act of the utterance (DA).…”

“…We consider two methods to represent previous turns: the actual utterance (text), and the dialog act of the utterance (DA). For each method, the most recent segmented sentence unit from each speaking party is considered as the history suggested by the length of context from Khatri et al (2018). We append the last segmented system unit (sys_unit), the previous segmented user unit (user_prev), and the current segmented user unit (user_cur) as sys_unit <u_p> user_prev <u_c> user_cur where <u_p> and <u_c> are special tokens to separate utterances.…”

“…Moreover, a human-machine dialog act scheme is not simply used to understand the dialog flow but also to help dialog systems plan their Table 1: Comparison of different major dialog act schemes. TOPIC refers to the dialog act designed for topic modeling from Khatri et al (2018). "?"…”

Contextual Dialogue Act Classification for Open-Domain Conversational Agents

“…Finally, we trained the overall model with an Adams optimizer and a learning rate of 0.001. All experiments for ADAN were conducted using the topic classifier API made available to the teams by the Amazon Alexa Prize [10]. To train the FastText model 20 , character 5-grams with word embedding of size 300 were used.…”