Recognition of Dialogue Acts in Multiparty Meetings Using a Switching DBN

Abstract

Abstract-This paper is concerned with the automatic recognition of Dialogue Acts (DAs) in multiparty conversational speech. We present a joint generative model for DA recognition in which segmentation and classification of DAs are carried out in parallel.Our approach to DA recognition is based on a switching dynamic Bayesian network (DBN) architecture. This generative approach models a set of features, related to lexical content and prosody, and incorporates a weighted interpolated factored language model. The switching DBN coordinates the recognition process by integrating the component models. The factored language model, which is estimated from multiple conversational data corpora, is used in conjunction with additional task specific language models. In conjunction with this joint generative model, we have also investigated the use of a discriminative approach, based on conditional random fields, to perform a reclassification of the segmented DAs.We have carried out experiments on the AMI corpus of multimodal meeting recordings, using both manually transcribed speech, and the output of an automatic speech recogniser, and using different configurations of the generative model. Our results indicate that the system performs well both on reference and fully automatic transcriptions. A further significant improvement in recognition accuracy is obtained by the application of the discriminative reranking approach based on conditional random fields.

Using Context Information for Dialog Act Classification in DNN Framework

“…The observation probabilities are obtained by DA specific wordbased language models, and a DA tag based ngram language model provides the transition probabilities between the DA tags. (Ji and Bilmes, 2005;Dielmann and Renals, 2008) used DBN for sequence decoding and examined both the generative and the conditional modeling approaches. CRF, as a powerful sequence labeling method, has also been widely used to incorporate context information for DA classification (Kim et al, 2010;Quarteroni et al, 2011;Chen and Eugenio, 2013;Dielmann and Renals, 2008).…”

“…(Ji and Bilmes, 2005;Dielmann and Renals, 2008) used DBN for sequence decoding and examined both the generative and the conditional modeling approaches. CRF, as a powerful sequence labeling method, has also been widely used to incorporate context information for DA classification (Kim et al, 2010;Quarteroni et al, 2011;Chen and Eugenio, 2013;Dielmann and Renals, 2008). It is worth noting that (Ribeiro et al, 2015) used different configurations to capture information from previous context in the SVM classifiers, such as n-grams or DA predictions.…”

Automatic dialogue act recognition with syntactic features

“…They use word sequence and pause duration as features. The authors of [30] exploit a Switching Dynamic Bayesian Network for segmentation, cascaded with a Conditional Random Field for dialogue act classification, while [31] jointly segments and tags with a single model.…”

“…These include Hidden Markov Models [11], Bayesian Networks [32], Discriminative Dynamic Bayesian Networks [33], BayesNet [28], Memory-based [34] and Transformation-based Learning [35], Decision Trees [36], Neural Networks [37], but also more advanced approaches such as Boosting [38], Latent Semantic Analysis [39], Hidden Backoff Models [40], Maximum Entropy Models [41], Conditional Random Fields [31,30] and Triangular-chain CRF [42].…”

Classification of patient case discussions through analysis of vocalisation graphs

“…More recently, several variants of topic identification methods have been applied to the analysis of meeting data. These include, among other things, detection of group actions [24], dialogue acts [10] and salient events such as "decisions" [15]. A related task is that of segmenting meetings recordings into topics [14,28,22].…”