DBN Based Joint Dialogue Act Recognition of Multiparty Meetings

Abstract

Joint Dialogue Act segmentation and classification of the new AMI meeting corpus has been performed through an integrated framework based on a switching dynamic Bayesian network and a set of continuous features and language models. The recognition process is based on a dictionary of 15 DA classes tailored for group decision-making. Experimental results show that a novel interpolated Factored Language Model results in a low error rate on the automatic segmentation task, and thus good recognition results can be achieved on AMI multiparty conversational speech.

Recognition of Dialogue Acts in Multiparty Meetings Using a Switching DBN

“…These experiments extend our previously published results in which an early version of the switching DBN model, without the use of interpolated FLMs, was used for DA recognition on the ICSI meetings corpus [24], and experiments on the AMI corpus using manual transcriptions only [21]. Our initial experiments, applying the complete framework to the 5 DA ICSI task, validates the methodology on an established task, forming the base for our investigations on the novel 15 DA AMI task.…”

“…In this work, we treat the segmentation and classification problems jointly and the process is coordinated by a switching DBN model [19], implemented using the Graphical Model ToolKit (GMTK) [20]. Figure 2 depicts the switching DBN model [21]. The transcribed words are represented as the sequence of discrete observable nodes W 0 , .…”

Recognition and understanding of meetings the AMI and AMIDA projects

“…Much existing work in this area is concerned with the extraction of content from written language; a major focus of AMI has been the extension of textual approaches to multimodal settings, involving the use of prosodic, video and contextual features. Our work in this area has included the development of automatic approaches to the segmentation and classification of phenomena such as dialogue acts [29], topics [30], and dominance and influence [31], as well as abstractive and extractive summarization [32] and content-based automatic camera selection [33]. Using the AMI corpus for all tasks, we have been able to agree on evaluation measures and procedures that allow us to compare different approaches and techniques, both internally and externally.…”

“…The segmentation problem is non-trivial, since a single stretch of speech (with no pauses) from a speaker may comprise several dialogue acts-and conversely a single dialogue act may contain pauses. Our approach to dialogue act recognition is based on a switching dynamic Bayesian network architecture which models a set of features related to lexical content and prosody and incorporates a weighted interpolated factored language model [29]. The switching DBN coordinates the recognition process by integrating all the available resources.…”

Detecting Action Items in Meetings

“…In these experiments we employed a manual DA segmentation, although automatic approaches are available [3].…”