Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech

Abstract

We describe a statistical approach for modeling dialogue acts in conversational speech, i.e., speech-act-like units such as STATEMENT, QUESTION, BACKCHANNEL, AGREEMENT, DIS-AGREEMENT, and APOLOGY. Our model detects and predicts dialogue acts based on lexical, collocational, and prosodic cues, as well as on the discourse coherence of the dialogue act sequence. The dialogue model is based on treating the discourse structure of a conversation as a hidden Markov model and the individual dialogue acts as observations emanating from the model states. Constraints on the likely sequence of dialogue acts are modeled via a dialogue act n-gram. The statistical dialogue grammar is combined with word n-grams, decision trees, and neural networks modeling the idiosyncratic lexical and prosodic manifestations of each dialogue act. We develop a probabilistic integration of speech recognition with dialogue modeling, to improve both speech recognition and dialogue act classification accuracy. Models are trained and evaluated using a large hand-labeled database of 1,155 conversations from the Switchboard corpus of spontaneous human-to-human telephone speech. We achieved good dialogue act labeling accuracy (65% based on errorful, automatically recognized words and prosody, and 71% based on word transcripts, compared to a chance baseline accuracy of 35% and human accuracy of 84%) and a small reduction in word recognition error.

Statistical framework for a Spanish spoken dialogue corpus

“…Webb et al (2005) make use of word N-grams of utterances for DA classi-3 fication on segmented turns. With this simple approach, they report results similar to Stolcke et al (2000) for the same corpus. Also with the same data, a relative improvement of 12% in performance is obtained by Rangarajan et al (2007) using maximum entropy modelling with prosodic, lexical and syntactic features.…”

“…In order to estimate this posterior probability, we consider two alternatives: the application of Bayes' rule to express the posterior probability in terms of much more straightforward models (Stolcke et al (2000), Young (2000)) or the application of transducers learnt by Grammatical Inference techniques (Vidal (1994)). These alternatives are reflected in two different models that provide two different methods of solving the optimisation problem of expression (1).…”

“…A language model is required to determine the probability of the DA sequences, and weight parameters should be used to balance the influence of the emitting and the language model. This type of model has previously been used to automatically annotate dialogues with DA, but the reported experiments assumed the availability of the segmentation of dialogue turns into utterances (Stolcke et al (2000)). Our proposal is to apply this type of model on unsegmented dialogue turns.…”

“…Similar to other NLP problems (like speech recognition and understanding, or statistical machine translation), data-based approaches to dialogue modelling 2 such as Stolcke et al (2000) and Young (2000) have been developed in the last decade. These machine learning approaches rely on statistical models that can be automatically estimated from annotated data, which in this case, are dialogues from the task (knowledge domain).…”

“…Levin et al (1999) try to find the most likely speech act sequence from the words of an utterance for the purpose of predicting more abstract labels called dialogue games. In Stolcke et al (2000), the purpose of labelling is to improve the speech recognition performance of the utterance, and it is applied to segmented turns. Both of them make use of a combination of N-grams and Hidden Markov models, but they report experiments over different corpora (CallHome Spanish and SwitchBoard), obtaining 56% and 70% accuracy in DA labelling, respectively.…”