Automatic Meeting Segmentation Using Dynamic Bayesian Networks

Abstract

Abstract-Multiparty meetings are a ubiquitous feature of organizations, and there are considerable economic benefits that would arise from their automatic analysis and structuring. In this paper, we are concerned with the segmentation and structuring of meetings (recorded using multiple cameras and microphones) into sequences of group meeting actions such as monologue, discussion and presentation. We outline four families of multimodal features based on speaker turns, lexical transcription, prosody, and visual motion that are extracted from the raw audio and video recordings. We relate these low-level features to more complex group behaviors using a multistream modelling framework based on multistream dynamic Bayesian networks (DBNs). This results in an effective approach to the segmentation problem, resulting in an action error rate of 12.2%, compared with 43% using an approach based on hidden Markov models. Moreover, the multistream DBN developed here leaves scope for many further improvements and extensions.

Performance of hidden Markov model and dynamic Bayesian network classifiers on handwritten Arabic word recognition

“…BNs are directed probabilistic graphical models. The random variables are represented by nodes, and the conditional dependences among the variables are represented by the arcs between the nodes [34]. BNs are graphical structures that represent the probabilistic relationships among large number of variables.…”

Privacy-sensitive recognition of group conversational context with sociometers

“…Regarding group behavior modeling, Zhang et al [50] and Dielmann et al [6] characterized group activities employing layered sequential approaches (either HMM or DBN), where the first layer modeled the individuals' behavior, and the second layer the activity (monologue, presentations, or discussions). Otsuka et al [35] described group activities in terms of conversational regimes (convergence or monologue, dyad-link and divergence).…”

“…Otsuka et al [35] described group activities in terms of conversational regimes (convergence or monologue, dyad-link and divergence). The work in [6,50] employed speaking-activity and motionactivity in terms of blobs (region of image pixels) as the features, whereas in [35] speaking-activity and visual gaze were used. The latter work was also extended to estimate interpersonal influence, interactivity, and centrality [36].…”

Automatic analysis of multiparty meetings

“…A major commonality that has been found across all these levels of segmentation is the need for multimodal features: text (speech transcriptions), prosody (timing and intonation), various features characterising interaction patterns, and various video features. As a specific example we describe a study in which "meeting actions" are automatically detected using from multiple streams of multimodal features (Dielmann and Renals 2007).…”