Audio-visual multi-person tracking and identification for smart environments

Bernardin, Keni; Stiefelhagen, Rainer

doi:10.1145/1291233.1291388

Cited by 31 publications

(26 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this context, robust and accurate tracking results have been demonstrated, e.g. [2,3], and current goals are to recover the pose of objects in addition to their localization, exploit other modalities such as audio [4], and characterize people activities. Another set of environement are open spaces.…”

Section: Key Factors and Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Person Bayesian Tracking with Multiple Cameras

Yao

Odobez

2009

Multi-Camera Networks

View full text Add to dashboard Cite

Object tracking is an important task within the field of computer vision, which is driven by the need to detect interesting moving objects in order to analyze and recognize their behaviours and activities. However, tracking multiple object is a complex task due to a large number of issues number ranging from the different types of sensing set-up to the complexity of the object appearance and behaviours.In this chapter, we analyze some of the important issues to solve for multiple object tracking, reviewing briefly how they are addressed in the literature. We then present a state-of-the-art algorithm for the tracking of a variable number of 3D persons in a multi-camera setting with partial field-of-view overlap. The algorithm illustrates how in a Bayesian framework the raised issues can be formulated and handled. More specifically, the tracking problem relies on a joint multi-object state space formulation with individual object states defined in the 3D world. It involves several key features for efficient and reliable tracking like the definition of appropriate multiobject dynamics and a global multi-camera observation model based on color and foreground measurements, the use of the Reversible-Jump Markov Chain Monte Carlo (RJ-MCMC) framework for efficient optimization, the exploitation of powerful human detector outputs in the MCMC proposals to automatically initialize/update object tracks. Experimental results on challenging real-world tracking sequences and situations demonstrate the efficiency of such an approach.

show abstract

Section: Key Factors and Related Workmentioning

confidence: 99%

“…Alternatively, one can represent people using sets of local templates or patches and geometric information, as proposed by Leibe et al [29]. In addition, other modalities such as audio microphone arrays can be used for localization, especially in smart rooms [4].…”

Section: Key Factors and Related Workmentioning

confidence: 99%

Multi-Person Bayesian Tracking with Multiple Cameras

Yao

Odobez

2009

Multi-Camera Networks

View full text Add to dashboard Cite

show abstract

“…More recently Bernardin and Stiefelhagen [2] have implemented a system for the simultaneous tracking and incremental multimodal identification of multiple users in a smart environment which fuses person track information, localized speaker ID and high definition visual ID cues opportunistically to gradually refine the global scene model and thus increase the system's confidence in the set of recognized identities. The improvements of combining acoustic features and 2D face images for identification of participants in a smart room environment are discussed in [15].…”

Section: Biometrics In Smart Environmentsmentioning

confidence: 99%

Integrating recognition and reasoning in smart environments

Menon¹,

Jayaraman²,

Govindaraju³

2008

4th International Conference on Intelligent Environments (IE 08)

View full text Add to dashboard Cite

Our goal is to develop 'smart indoor environments' that are monitored unobtrusively by biometric capture devices, such as video cameras, microphones, etc. Such environments will keep track of their occupants and be capable of answering queries about the occupants' whereabouts. In order to develop a unified model that is applicable across diverse biometric modalities, we propose an abstract state transition framework in which different recognition steps are abstracted by events, and the reasoning necessary to effect state transitions is abstracted by a transition function. We define the metrics of 'precision' and 'recall' of a smart environment to evaluate how well it tracks its occupants. We show how the overall performance of our smart environment can be improved through the use of spatiotemporal knowledge of the environment. A prototype based upon our proposed abstract framework indicates that integrating recognition and reasoning capabilities substantially improves the overall performance of the environment

show abstract

“…In this context, robust and accurate tracking results have been demonstrated, e.g. [Bernardin et al, 2006, Fleuret et al, 2008, and current goals consists of improving tracking robustness under higher crowding levels, recovering the pose of objects in addition to their localization, exploiting other modalities such as audio [Bernardin and Stiefelhagen, 2007], and characterizing people activities.…”

Section: Introductionmentioning

confidence: 99%

Sampling techniques for audio-visual tracking and head pose estimation

Odobez¹,

Lanz²

2012

Multimodal Signal Processing

View full text Add to dashboard Cite

Analyzing people behaviors in smart environment using multimodal sensors requires to answer a set of typical questions: who are the people, where are they, what activities are they doing, when, with whom are they interacting, and how. In this view, locating people or their faces and characterizing them (e.g. extracting their body or head orientation) allows to address the first two questions (who and where), and is usually one of the first steps before applying higher level multimodal scene analysis algorithms that address the other questions. In the last ten years, tracking algorithms have experienced considerable progresses, particularly in indoor environment or for specific applications, where they have reached a maturity allowing their deployment in real systems and applications. Nevertheless, there are still several issues that can make the tracking difficult: background clutter, potentially small object size; complex shape, appearance, and motion, and their changes over time or across camera views; inaccurate/rough scene calibration or inconsistent camera calibration between views for 3D tracking; real-time processing requirements. In what follows, we discuss some important aspects of tracking algorithms, and ultimately introduce the remaining of the chapter content.Scenarios and Set-ups. Scenarios and application needs strongly influence the considered physical environment, and therefore the set-up (where, how many, and what type of sensors are used) and choice of a tracking 84

show abstract

Audio-visual multi-person tracking and identification for smart environments

Cited by 31 publications

References 23 publications

Multi-Person Bayesian Tracking with Multiple Cameras

Multi-Person Bayesian Tracking with Multiple Cameras

Integrating recognition and reasoning in smart environments

Sampling techniques for audio-visual tracking and head pose estimation

Contact Info

Product

Resources

About