Kalman filters for audio-video source localization

Gehrig, Tobias; Nickel, Kai; Ekenel, Hazım Kemal; Klee, Ulrich; McDonough, John

doi:10.1109/aspaa.2005.1540183

Cited by 49 publications

(38 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Data association algorithms with Bayesian methods and PHD filter in target tracking applications can be found in [7], [29], [30], [31] and [32]. However, some researchers found that classical data association algorithms are computationally expensive, and this led them to fuse multi-modal measurements inside their proposed framework [11], [14], [16], [17], [20] as we also do here.…”

Section: Introductionmentioning

confidence: 94%

“…At every iteration, after the comparison of γ av k with γ k , S k and σ 2 k values are updated using (16) in order to find the optimal N k by (14). The last step of the PF algorithm is resampling and since the N k value has just been changed, this step is also modified for the new N k .…”

Section: Improved Av Tracking With Adaptive Particle Filtermentioning

confidence: 99%

“…A popular approach for tracking speakers with audio and video data is to use a state-space approach based on the Bayesian framework, for example, the Kalman filter (KF) for linear motion and sensor models [10], extensions of KF for the nonlinear models using the first order Taylor expansion including the decentralized Kalman filter (DKF) [11], [12] and extended Kalman filter (EKF) [13], [14], and the particle filter (PF) for nonlinear and non-Gaussian models [15]. In comparison to the KF and EKF approaches, the PF approach is more robust for nonlinear models as it can approach the Bayesian optimal estimate with a sufficiently large number of particles [15].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Audio Assisted Robust Visual Tracking With Adaptive Particle Filtering

Kılıç

Barnard

Wang

et al. 2015

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Abstract-The problem of tracking multiple moving speakers in indoor environments has recently received much attention. Earlier techniques were based purely on vision, but the theoretical and algorithmic advances and a constant growth in speed of processing have led to the emergence of techniques which allow the fusion of audio and visual data. The fusion of multi-modal information has been shown to be instrumental in improving tracking performance, as well as robustness in the case of challenging situations like occlusions (by the limited field of view of cameras or by other speakers). However, data fusion algorithms often suffer from noise corrupting the sensor measurements which cause non-negligible detection errors. Here, a novel approach to combining audio and visual data is proposed. In our framework, we employ audio data as an aid to particle filter (PF) based visual tracking, by using the direction of arrival angles of the audio sources to reshape the typical Gaussian noise distribution of particles in the propagation step and to weight the observation model in the measurement step. This approach is further improved by solving a typical problem associated with the PF. It has been observed that the efficiency and accuracy of the PF usually depend on the number of particles and noise variance used in the estimation and propagation functions for re-allocating these particles at each iteration. Both of these parameters are specified beforehand and are kept fixed in the regular implementation of the PF which makes the tracker unstable in practice. To address these problems, we design an algorithm which adapts both the number of particles and noise variance based on tracking error and the area occupied by the particles in the image. Experiments on the AV 16.3 dataset show the advantage of our proposed methods over the baseline PF method and an existing adaptive PF algorithm for tracking occluded speakers with a significantly reduced number of particles.Index Terms-Audio-visual speaker tracking, particle filter, adaptive particle filter.

show abstract

Section: Introductionmentioning

confidence: 94%

Section: Improved Av Tracking With Adaptive Particle Filtermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Audio Assisted Robust Visual Tracking With Adaptive Particle Filtering

Kılıç

Barnard

Wang

et al. 2015

IEEE Trans. Multimedia

View full text Add to dashboard Cite

show abstract

“…Face and upper body parts can be detected using contour extraction by performing edges and motion analysis and then combined with audio detection in particle filter framework ( [1], [11], [40]). Gehrig et al ( [12]) apply audio detection to generate face positions that could also be observed by multiple cameras.…”

Section: Related Workmentioning

confidence: 99%

“…A significant amount of work has been reported on detecting and tracking single or multiple moving objects using Kalman filter (KF) ( [12], [13]), particle filter (PF) ( [3], [14], [15]) and variants of probabilistic data association (PDA) ( [16], [17]). Multimodal multi-sensor configurations are used for object tracking ( [14], [18], [19], [20]) to compensate for failure of each modality.…”

Section: Related Workmentioning

confidence: 99%

Target Detection and Tracking With Heterogeneous Sensors

Zhou

Taj

Cavallaro

2008

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

Abstract-We present a multimodal detection and tracking algorithm for sensors composed of a camera mounted between two microphones. Target localization is performed on color-based change detection in the video modality and on Time Difference of Arrival (TDOA) estimation between the two microphones in the audio modality. The TDOA is computed by multi-band Generalized Cross Correlation (GCC) analysis. The estimated directions of arrival are then post-processed using a Riccati Kalman filter. The visual and audio estimates are finally integrated, at the likelihood level, into a particle filter (PF) that uses a zero-order motion model, and a Weighted Probabilistic Data Association (WPDA) scheme. We demonstrate that the Kalman filtering (KF) improves the accuracy of the audio source localization and that the WPDA helps to enhance the tracking performance of sensor fusion in reverberant scenarios. The combination of multi-band GCC, KF and WPDA within the particle filtering framework improves the performance of the algorithm in noisy scenarios. We also show how the proposed audiovisual tracker summarizes the observed scene by generating metadata that can be transmitted to other network nodes instead of transmitting the raw images and can be used for very low bit rate communication. Moreover, the generated metadata can also be used to detect and monitor events of interest.Index Terms-Multimodal detection and tracking, Kalman filter, particle filter, heterogeneous sensors, low bit rate communication.

show abstract

Multimodal Identity Tracking in a Smartroom

Bernardin¹,

Ekenel²,

Stiefelhagen³

IFIP International Federation for Information Processing

View full text Add to dashboard Cite

Abstract. The automatic detection, tracking, and identification of multiple people in intelligent environments is an important building block on which smart interaction systems can be designed. Those could be, e.g. gesture recognizers, head pose estimators or far field speech recognizers and dialog systems. In this paper, we present a system which is capable of tracking multiple people in a smartroom enviromnent while infering their identities in a completely automatic and imobtrusive way. It relies on a set of fixed and active cameras to track the users and get closeups of their faces for identification, and on several microphone arrays to determine active speakers and steer the attention of the system. Information coming asynchronously from several sources, such as position updates from audio or visual trackers and identification events from identification modules, is fused at higher level to gradually refine the room's situation model. The system has been trained on a smaU set of users and showed good performance at acquiring and keeping their identities in a smart room environment.

show abstract

Kalman filters for audio-video source localization

Cited by 49 publications

References 9 publications

Audio Assisted Robust Visual Tracking With Adaptive Particle Filtering

Audio Assisted Robust Visual Tracking With Adaptive Particle Filtering

Target Detection and Tracking With Heterogeneous Sensors

Multimodal Identity Tracking in a Smartroom

Contact Info

Product

Resources

About