Multimodal Integration for Meeting Group Action Segmentation and Recognition

Al-Hames, Marc; Dielmann, Alfred; Gática-Pérez, Daniel; Reiter, S.; Renals, Steve; Rigoll, Gerhard; Zhang, Dong

doi:10.1007/11677482_5

Cited by 16 publications

(12 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The prosodic features that we used in the generative model (with the exception of F0 variance) were discretised and used in conjunction with the lexical information during the CRF re-labeling process, implemented with CRF++ 4 . Table VI reports the recognition performances after discriminative re-classification.…”

Section: Discriminative Re-classification Of Joint Recognition Oumentioning

confidence: 99%

“…Example applications have included automatic summarisation [1], topic segmentation and labelling [2], [3], group action detection [4], [5], [6], participant influence [7], and dialog structure annotation [8]. The reliable recognition of the DA sequence in a meeting, and the resulting knowledge of the discourse structure, plays an important role in the development of such applications.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Recognition of Dialogue Acts in Multiparty Meetings Using a Switching DBN

Dielmann

Renals

2008

IEEE Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Abstract-This paper is concerned with the automatic recognition of Dialogue Acts (DAs) in multiparty conversational speech. We present a joint generative model for DA recognition in which segmentation and classification of DAs are carried out in parallel.Our approach to DA recognition is based on a switching dynamic Bayesian network (DBN) architecture. This generative approach models a set of features, related to lexical content and prosody, and incorporates a weighted interpolated factored language model. The switching DBN coordinates the recognition process by integrating the component models. The factored language model, which is estimated from multiple conversational data corpora, is used in conjunction with additional task specific language models. In conjunction with this joint generative model, we have also investigated the use of a discriminative approach, based on conditional random fields, to perform a reclassification of the segmented DAs.We have carried out experiments on the AMI corpus of multimodal meeting recordings, using both manually transcribed speech, and the output of an automatic speech recogniser, and using different configurations of the generative model. Our results indicate that the system performs well both on reference and fully automatic transcriptions. A further significant improvement in recognition accuracy is obtained by the application of the discriminative reranking approach based on conditional random fields.

show abstract

Section: Discriminative Re-classification Of Joint Recognition Oumentioning

confidence: 99%

mentioning

confidence: 99%

Recognition of Dialogue Acts in Multiparty Meetings Using a Switching DBN

Dielmann

Renals

2008

IEEE Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…In this work, the sub-actions have no obvious interpretation, and their number is a model parameter learned during training or set by hand, which makes the structure of the models more difficult to interpret. An initial comparison of various recognition models on the same task, including the layered HMM, the multilevel DBN, and other approaches, was presented by Al-Hames et al [1].…”

Section: Turn-taking Patternsmentioning

confidence: 99%

Analyzing Group Interactions in Conversations: a Review

Gática-Pérez

2006

2006 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems

Self Cite

View full text Add to dashboard Cite

Abstract-Multiparty face-to-face conversations in professional and social settings represent an emerging research domain for which automatic activity-based analysis is relevant for scientific and practical reasons. The activity patterns emerging from groups engaged in conversations are intrinsically multimodal and thus constitute interesting target problems for multistream and multisensor fusion techniques. In this paper, a summarized review of the literature on automatic analysis of group activities in face-to-face conversational settings is presented. A basic categorization of group activities is proposed based on their typical temporal scale, and existing works are then discussed for various types of activities and trends including addressing, turn taking, interest, and dominance.

show abstract

“…Third, the best I-HMM model is the asynchronous HMM (a model that explicitly accounts for variations of alignment between two data streams), which suggests that some asynchrony exists for the defined group actions, and that such asynchrony is reasonably captured by the model. A recent comparison of the layered HMM and other models on the same task appears in [1]. …”

Section: Modeling Group Interaction With Layersmentioning

confidence: 99%

“…testify. Very broadly speaking, one can identify three different -fortunately not exclusive-positions regarding what is relevant about meeting collections: (1) what they are; (2) what can be done with them; and (3) what needs to be done with them. The first view acknowledges that meetings by themselves are relevant insofar as they constitute an expression of human interaction, the field of study of more than one branch of science [6,25].…”

Section: Introductionmentioning

confidence: 99%

Extracting information from multimedia meeting collections

Gática-Pérez

Zhang

Bengio

2005

Proceedings of the 7th ACM SIGMM International Workshop on Multimedia Information Retrieval

Self Cite

View full text Add to dashboard Cite

Multimedia meeting collections, composed of unedited audio and video streams, handwritten notes, slides, and electronic documents that jointly constitute a raw record of complex human interaction processes in the workplace, have attracted interest due to the increasing feasibility of recording them in large quantities, by the opportunities for information access and retrieval applications derived from the automatic extraction of relevant meeting information, and by the challenges that the extraction of semantic information from real human activities entails. In this paper, we present a succint overview of recent approaches in this field, largely influenced by our own experiences. We first review some of the existing and potential needs for users of multimedia meeting information systems. We then summarize recent work on various research areas addressing some of these requirements. In more detail, we describe our work on automatic analysis of human interaction patterns from audio-visual sensors, discussing open issues in this domain.

show abstract

Multimodal Integration for Meeting Group Action Segmentation and Recognition

Cited by 16 publications

References 13 publications

Recognition of Dialogue Acts in Multiparty Meetings Using a Switching DBN

Recognition of Dialogue Acts in Multiparty Meetings Using a Switching DBN

Analyzing Group Interactions in Conversations: a Review

Extracting information from multimedia meeting collections

Contact Info

Product

Resources

About