Multimodal speaker detection using error feedback dynamic Bayesian networks

Pavlović, Vladimir; Garg, Ashutosh; Rehg, James M.; Huang, Thomas S.

doi:10.1109/cvpr.2000.854730

Cited by 37 publications

(42 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are a few methods proposed to solve the audio-visual source localization problem for multimedia [32], [33], but they mostly assume that the sound source is a human speaker's mouth. Thus, they cannot be applied to general multimedia content that contain non-speech audio signal.…”

Section: Foveated Video Coding Based On Audio-visual Focus Of Atmentioning

confidence: 99%

Subjective Quality Evaluation of Foveated Video Coding Using Audio-Visual Focus of Attention

Lee

Simone

Ebrahimi

2011

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

Abstract-This paper presents a foveated coding method using audio-visual focus of attention and its evaluation through extensive subjective experiments on both standard-definition and high-definition sequences. Regarding a sound-emitting region as the location drawing the human attention, the method applies varying quality levels in an image frame according to the distance of a pixel to the identified sound source. Two experiments are presented to prove the efficiency of the method. Experiment 1 examines the validity and effectiveness of the method in comparison to the constant quality coding for high-quality conditions. In Experiment 2, the method is compared to the fixed bit rate coding for low quality conditions where coding artifacts are noticeable. The results demonstrate that the foveated coding method provides considerable coding gain without significant quality degradation, but uneven distributions of the coding artifacts (blockiness) by the method are often less preferred than the uniform distribution of the artifacts. Additional interesting findings are also discussed, such as content dependence of the performance of the method, the memory effect in multiple viewings, and the difference in the quality perception for frame size variations.Index Terms-Audio-visual focus of attention, content dependence, foveated coding, H.264/AVC, memory effect, quality of experience, subjective quality assessment.

show abstract

Section: Foveated Video Coding Based On Audio-visual Focus Of Atmentioning

confidence: 99%

Subjective Quality Evaluation of Foveated Video Coding Using Audio-Visual Focus of Attention

Lee

Simone

Ebrahimi

2011

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

show abstract

“…Localizing and tracking speakers in enclosed spaces using AV information has increasingly attracted attention in signal processing and computer vision [36,17,7,34,13,43,48,1,3,6,5], given the complementary characteristics of each modality. Broadly speaking, the differences among existing works arise from the overall goal (tracking single vs. multiple speakers), the specific detection/tracking framework, and the AV sensor configuration.…”

Section: Related Workmentioning

confidence: 99%

“…Broadly speaking, the differences among existing works arise from the overall goal (tracking single vs. multiple speakers), the specific detection/tracking framework, and the AV sensor configuration. Much work has concentrated on the single-speaker case, assuming either single-person scenes [7,34,1], or multiperson scenes where only the location of the current speaker needs to be tracked [36,17,13,43,48,3]. Many of these works used simple sensor configurations (e.g.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Audiovisual Probabilistic Tracking of Multiple Speakers in Meetings

Gática-Pérez

Lathoud

Odobez

et al. 2007

IEEE Trans. Audio Speech Lang. Process.

120

116

View full text Add to dashboard Cite

Abstract. Tracking speakers in multiparty conversations constitutes a fundamental task for automatic meeting analysis. In this paper, we present a probabilistic approach to jointly track the location and speaking activity of multiple speakers in a multisensor meeting room, equipped with a small microphone array and multiple uncalibrated cameras. Our framework is based on a mixed-state dynamic graphical model defined on a multiperson state-space, which includes the explicit definition of a proximity-based interaction model. The model integrates audio-visual (AV) data through a novel observation model. Audio observations are derived from a source localization algorithm. Visual observations are based on models of the shape and spatial structure of human heads. Approximate inference in our model, needed given its complexity, is performed with a Markov Chain Monte Carlo particle filter (MCMC-PF), which results in high sampling efficiency. We present results -based on an objective evaluation procedure-that show that our framework (1) is capable of locating and tracking the position and speaking activity of multiple meeting participants engaged in real conversations with good accuracy; (2) can deal with cases of visual clutter and partial occlusion; and (3) significantly outperforms a traditional sampling-based approach. IDIAP-RR 05-27

show abstract

“…Most existing methods for speaker detection are realized by combining techniques of sound localization via a microphone array and human tracking via background subtraction by using coupled Hidden Markov Models (HMMs) or Dynamic Bayesian Networks (DBNs) [11,2]. However, because of the spatial resolution of the microphone array, these methods can become ineffective in situations where speakers are physically close to each other.…”

Section: Introductionmentioning

confidence: 99%