UPC Audio, Video and Multimodal Person Tracking Systems in the Clear Evaluation Campaign

Abad, Alberto; Canton-­Ferrer, Cristian; Segura, Carlos; Landabaso, J.L.; Macho, Dušan; Casas, Josep R.; Hernando, Javier; Pardàs, Montse; Nadeu, Climent

doi:10.1007/978-3-540-69568-4_6

Cited by 9 publications

(9 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In indoor scenarios, active speaker localization and tracking in meeting rooms can be done by using audiovisual fusion where audio localization is done using beamforming and visual target is tracked using Kalman filter [7] or by using TDOA with Particle filter [8]. Tracking using audio modality is strongly effected by reverberation.…”

Section: Related Workmentioning

confidence: 99%

Audio-assisted trajectory estimation in non-overlapping multi-camera networks

Taj

Cavallaro

2009

2009 IEEE International Conference on Acoustics, Speech and Signal Processing

View full text Add to dashboard Cite

We present an algorithm to improve trajectory estimation in networks of non-overlapping cameras using audio measurements. The algorithm fuses audiovisual cues in each camera's field of view and recovers trajectories in unobserved regions using microphones only. Audio source localization is performed using Stereo Audio and Cycloptic Vision (STAC) sensor by estimating the time difference of arrival (TDOA) between microphone pair and then by computing the cross correlation. Audio estimates are then smoothed using Kalman filtering. The audio-visual fusion is performed using a dynamic weighting strategy. We show that using a multi-modal sensor with combined visual (narrow) and audio (wider) field of view can enable extended target tracking in non-overlapping camera settings. In particular, the weighting scheme improves performance in the overlapping regions. The algorithm is evaluated in several multi-sensor configurations using synthetic data and compared with state of the art algorithm.

show abstract

Section: Related Workmentioning

confidence: 99%

Audio-assisted trajectory estimation in non-overlapping multi-camera networks

Taj

Cavallaro

2009

2009 IEEE International Conference on Acoustics, Speech and Signal Processing

View full text Add to dashboard Cite

show abstract

“…The problem of audiovisual tracking involves the estimation of the arrival angle of the audio signal, video detection, filtering and smoothing of the two modalities, fusion and [36] Camera and 2 microphones TDNN Surveillance [1], [11], [15], [32] PF Surveillance and teleconferencing [12], [13], [37] KF, DKF Smart rooms [38] Multiple cameras and microphone arrays LDA Smart rooms [9], [30], [31], [39] PF Meeting rooms finally joint state estimation. Let the target state be defined as y(t) = (x, y, w, h, H), where (x, y) is the center of the ellipse approximating the object shape, (w, h) are the width and height of the bounding box and H is the color histogram of the object.…”

Section: Audiovisual Trackingmentioning

confidence: 99%

Target Detection and Tracking With Heterogeneous Sensors

Zhou

Taj

Cavallaro

2008

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

Abstract-We present a multimodal detection and tracking algorithm for sensors composed of a camera mounted between two microphones. Target localization is performed on color-based change detection in the video modality and on Time Difference of Arrival (TDOA) estimation between the two microphones in the audio modality. The TDOA is computed by multi-band Generalized Cross Correlation (GCC) analysis. The estimated directions of arrival are then post-processed using a Riccati Kalman filter. The visual and audio estimates are finally integrated, at the likelihood level, into a particle filter (PF) that uses a zero-order motion model, and a Weighted Probabilistic Data Association (WPDA) scheme. We demonstrate that the Kalman filtering (KF) improves the accuracy of the audio source localization and that the WPDA helps to enhance the tracking performance of sensor fusion in reverberant scenarios. The combination of multi-band GCC, KF and WPDA within the particle filtering framework improves the performance of the algorithm in noisy scenarios. We also show how the proposed audiovisual tracker summarizes the observed scene by generating metadata that can be transmitted to other network nodes instead of transmitting the raw images and can be used for very low bit rate communication. Moreover, the generated metadata can also be used to detect and monitor events of interest.Index Terms-Multimodal detection and tracking, Kalman filter, particle filter, heterogeneous sensors, low bit rate communication.

show abstract

“…Concerning the visual tracking algorithms, two main approaches have been followed by the various developed 3D tracking systems: First, a model-based approach where a 3D model of the tracked object is maintained by rendering it onto the camera views and searching for supporting evidence in each view to update its parameters [2,6,11,31,33,40]. Second, a data-driven approach where 2D trackers operate independently on the separate camera views and the 2D tracks belonging to a same target are collected into a 3D track [28,52,56].…”

Section: Person Trackingmentioning

confidence: 99%

“…On the acoustic side, approaches can be roughly categorized as follows: Approaches which rely on the computation of a more or less coarse global coherence field (GCF, or SRP-PHAT), on which the tracking of correlation peaks is performed [2,9]; particle filter approaches, which approximate the belief on speaker positions by a set of samples and measure the agreement of the observed acoustic signals (their correlation value) given each sample position hypothesis [39]; approaches that feed computed time delays of arrival (TDOAs) between microphone pairs directly as observations to a Kalman or other probabilistic tracker [24,30,50]. The best performing system was based on a Joint Probabilistic Data Association Filter (JPDAF), which keeps track of a number of sound sources, including noise sources, resolving data associations and position updates jointly for all tracks.…”

Section: Person Trackingmentioning

confidence: 99%

Tracking identities and attention in smart environments - contributions and progress in the CHIL project

Stiefelhagen

Bernardin

Ekenel

et al. 2008

2008 8th IEEE International Conference on Automatic Face &Amp; Gesture Recognition

View full text Add to dashboard Cite

To provide intelligent services in a smart environments it is necessary to acquire information about the room, the people in it and their interactions. This includes, for example, the number of people, their identities, locations, postures, body and head orientations, among others. This paper gives an overview of the perceptual technology evaluations that were conducted in the CHIL project, specifically those held in the CLEAR 2006 and 2007 evaluation workshops. We then summarize the main achievements and lessons learnt in the project in the areas of person tracking, person identification and head pose estimation, all of which are critical perception components in order to build perceptive smart environments.

show abstract

UPC Audio, Video and Multimodal Person Tracking Systems in the Clear Evaluation Campaign

Cited by 9 publications

References 7 publications

Audio-assisted trajectory estimation in non-overlapping multi-camera networks

Audio-assisted trajectory estimation in non-overlapping multi-camera networks

Target Detection and Tracking With Heterogeneous Sensors

Tracking identities and attention in smart environments - contributions and progress in the CHIL project

Contact Info

Product

Resources

About