Extracting information from multimedia meeting collections

Abstract

Multimedia meeting collections, composed of unedited audio and video streams, handwritten notes, slides, and electronic documents that jointly constitute a raw record of complex human interaction processes in the workplace, have attracted interest due to the increasing feasibility of recording them in large quantities, by the opportunities for information access and retrieval applications derived from the automatic extraction of relevant meeting information, and by the challenges that the extraction of semantic information from real human activities entails. In this paper, we present a succint overview of recent approaches in this field, largely influenced by our own experiences. We first review some of the existing and potential needs for users of multimedia meeting information systems. We then summarize recent work on various research areas addressing some of these requirements. In more detail, we describe our work on automatic analysis of human interaction patterns from audio-visual sensors, discussing open issues in this domain.

Automatic nonverbal analysis of social interaction in small groups: A review

“…Face-to-face conversations represent a fundamental case of social interaction as they are ubiquitous and constitute by far -despite the increased use of computed-mediated communication tools -the most natural, enjoyable, and effective way to fulfill our social needs. More specifically, the computational analysis of group conversations has an enormous value on their own for several social sciences [8,92], and could open doors to a number of relevant applications that support interaction and communication, including self-assessment, training and educational tools, and systems to support group collaboration [37,101,53,103], through the automatic sensing, analysis, and interpretation of social behavior.…”

Speakers Role Recognition in Multiparty Audio Recordings Using Social Network Analysis and Duration Distribution Modeling

“…Audio based information is very important also in the analysis of meeting recordings where speech is the main channel of communication between the various participants. Several works [3], [15], [16] are aimed at identifying group actions like discussions, agreement, and monologues. The proposed approach is based on machine learning algorithms fed with features (e.g., speaker location) extracted from both audio and visual channel.…”

“…The above expressions enable one to estimate the probability of an actor being the AM as follows: (16) where is the role of .…”

“…The second term of the product is estimated with (16) and the first one can be estimated by considering that the second anchorman is the speaker that interacts more than anybody else with the AM. In other words, if is the AM and is the SA, the value is expected to be, on average, higher than for .…”