Automatic analysis of multimodal group actions in meetings

McCowan, L.; Gática-Pérez, Daniel; Bengio, Samy; Lathoud, Guillaume; Barnard, Mark; Zhang, D.

doi:10.1109/tpami.2005.49

Cited by 289 publications

(215 citation statements)

References 47 publications

Supporting

Mentioning

211

Contrasting

Order By: Relevance

“…A few studies also included audio features [25], [26], [91] and found significant advantage in doing so. Multimodal multisource event detection is likely to receive more attention in the near future due to many emerging applications and the availability of large multimodal collections.…”

Section: ) Discussionmentioning

confidence: 99%

Event Mining in Multimedia Streams

2008

View full text Add to dashboard Cite

| Events are real-world occurrences that unfold over space and time. Event mining from multimedia streams improves the access and reuse of large media collections, and it has been an active area of research with notable recent progress. This paper contains a survey on the problems and solutions in event mining, approached from three aspects: event description, event-modeling components, and current event mining systems. We present a general characterization of multimedia events, motivated by the maxim of five BW[s and one BH[ for reporting real-world events in journalism: when, where, who, what, why, and how. We discuss the causes for semantic variability in real-world descriptions, including multilevel event semantics, implicit semantics facets, and the influence of context. We discuss five main aspects of an event detection system. These aspects are: the variants of tasks and event definitions that constrain system design, the media capture setup that collectively define the available data and necessary domain assumptions, the feature extraction step that converts the captured data into perceptually significant numeric or symbolic forms, statistical models that map the feature representations to richer semantic descriptions, and applications that use event metadata to help in different information-seeking tasks. We review current event-mining systems in detail, grouping them by the problem formulations and approaches. The review includes detection of events and actions in one or more continuous sequences, events in edited video streams, unsupervised event discovery, events in a collection of media objects, and a discussion on ongoing benchmark activities. These problems span a wide range of multimedia domains such as surveillance, meetings, broadcast news, sports, documentary, and films, as well as personal and online media collections. We conclude this survey with a brief outlook on open research directions.

show abstract

Section: ) Discussionmentioning

confidence: 99%

Event Mining in Multimedia Streams

2008

View full text Add to dashboard Cite

show abstract

“…In the literature, the modelling of human interactions has reached a deep maturity, both under a pure visual perspective [71,51], and also considering other modalities [60,4,46,15,45,14,10]. Anyway, like in the case of the definition of behavior, there is the lack of a common, formal definition of interaction.…”

Section: Surveillance and Monitoringmentioning

confidence: 99%

Socially intelligent surveillance and monitoring: Analysing social dimensions of physical space

Cristani

Murino

Vinciarelli

2010

2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops

View full text Add to dashboard Cite

In general terms, surveillance and monitoring technologies aim at understanding what people do in a given environment, whether this means to ensure the safety of workers on the factory floor, to detect crimes occurring in indoor or outdoor settings, or to monitor the flow of large crowds through public spaces. However, surveillance and monitoring technologies rarely consider that they analyze human behavior, a phenomenon subject to principles and laws rigorous enough to produce stable and predictable patterns corresponding to social, affective, and psychological phenomena. On the other hand, these phenomena are the subject of other computing domains, in particular Social Signal Processing and Affective Computing, that typically neglect scenarios relevant to surveillance and monitoring technologies, especially when it comes to social and affective dimensions of space in human activities. The goal of this paper is to show that the investigation of the overlapping area between surveillance and monitoring on one side, and Social Signal Processing and Affective Computing on the other side can bring significant progress in both domains and open a number of interesting research perspectives.

show abstract

“…One of the goals of the AMI and M4 projects is to find these kinds of group events in order to detect meeting structures (see e.g. [19]). To detect these highest level components a combination of the first and second level of interpreted information is needed.…”

Section: Meeting Modelling: Overviewmentioning

confidence: 99%

“…a presentation activity when observing a pattern of certain NCAs, CAs and physical state combined with certain argumentative structures in the data. The training can be done using machine learning techniques like Hidden Markov Models [19] or Dynamic Bayesian Networks, looking for statistical correlations.…”

Section: The Fourth Levelmentioning

confidence: 99%

Meeting Modelling in the Context of Multimodal Research

Reidsma

Rienks

Jovanović

2005

Machine Learning for Multimodal Interaction

View full text Add to dashboard Cite

Abstract. This paper presents a framework for corpus based multimodal research. Part of this framework is applied in the context of meeting modelling. A generic model for different aspects of meetings is discussed. This model leads to a layered description of meetings where each layer adds a level of interpretation for distinct aspects based on information provided by lower layers. This model should provide a starting point for selecting annotation schemes for layers of the meeting and for defining a hierarchy between individual layers.

show abstract

Automatic analysis of multimodal group actions in meetings

Cited by 289 publications

References 47 publications

Event Mining in Multimedia Streams

Event Mining in Multimedia Streams

Socially intelligent surveillance and monitoring: Analysing social dimensions of physical space

Meeting Modelling in the Context of Multimodal Research

Contact Info

Product

Resources

About