Audio-assisted trajectory estimation in non-overlapping multi-camera networks

Abstract

We present an algorithm to improve trajectory estimation in networks of non-overlapping cameras using audio measurements. The algorithm fuses audiovisual cues in each camera's field of view and recovers trajectories in unobserved regions using microphones only. Audio source localization is performed using Stereo Audio and Cycloptic Vision (STAC) sensor by estimating the time difference of arrival (TDOA) between microphone pair and then by computing the cross correlation. Audio estimates are then smoothed using Kalman filtering. The audio-visual fusion is performed using a dynamic weighting strategy. We show that using a multi-modal sensor with combined visual (narrow) and audio (wider) field of view can enable extended target tracking in non-overlapping camera settings. In particular, the weighting scheme improves performance in the overlapping regions. The algorithm is evaluated in several multi-sensor configurations using synthetic data and compared with state of the art algorithm.

Bayesian integration of audio and visual information for multi-target tracking using a CB-member filter

“…Figures 2-3 show snapshots of the tracking results in two of the sequences. 4 For the first sequence shown in in Fig. 2, we have shown the particle blobs as well as the final estimates.…”

“…However, the availability itself needs to be determined and can be erroneous in presence of clutter measurements or measurements corresponding to other speaking and visible targets. The most common solution is to linearly combine the measurement likelihoods of the visual and audio observations where the weights of the combination are adjusted dynamically according to an acoustic confidence measure [3] or using separate confidence measures for the audio and video channels [4]. However, linear combination of the two likelihoods is mainly heuristic and not mathematically accurate.…”

Design and Implementation of Enhanced Surveillance Platform with Low-Power Wireless Audio Sensor Network

“…But the number of synchronized sensor is usually limited by the maximal output current of one camera. In [10][11][12], authors employed a multi-modal detection and tracking algorithm. They localized the audio source by estimating the time difference of arrival and improved trajectory estimation in networks of nonoverlapping cameras using audio measurements.…”

Interaction recognition in wide areas using audiovisual sensors

“…In this case, as the microphone pair is unable to provide the localization information, we estimate the trajectory using a first-order motion model. Finally, we perform the audio-visual fusion within Kalman filtering with a weighted sum of the two measurements [10] that only penalizes audio detections in overlapping regions and gives a weight no smaller than 0.5 to the video modality, when available.…”