Adaptive Fusion of Speech and Lip Information for Robust Speaker Identification

Wark, Tim; Sridharan, Sridha

doi:10.1006/dspr.2001.0397

Cited by 48 publications

(35 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For audiovisual fusion, we adopted the score dispersion as an SCM approach in the baseline fusion scheme. Assuming K speakers, the fusion procedure is as follows [1], [2]. a) Generate the audio and video log-likelihood scores through individual classifiers for the input of AV fusion; …”

Section: Baseline Avsi System: Score-based Fusionmentioning

confidence: 99%

“…Multi-modal integration for audio-visual speaker identification (AVSI) is one of the robust approaches [1]- [5] in noisy environments, where speech signals have relatively high levels of distortion. The main issues concerning AVSI involve an integration structure and reliability decision-making.…”

Section: Introductionmentioning

confidence: 99%

“…In state-of-the-art bimodal biometrics technology, a number of approaches have been proposed for reliability measures, which can be broadly classified into signal-based measures (SIM) and score-based measures (SCM). SCMs utilize the distribution of the model probabilities [1]- [3], and in SIM, SNR and the voicing index are the representative parameters for audio signal [4], [5] only. However, until now there has been no SIM for the corresponding video signal in AVSI.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Visual Signal Reliability for Robust Audio-Visual Speaker Identification

Tariquzzaman

Kim

et al. 2011

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYIn this paper, a novel visual signal reliability (VSR) measure is proposed to consider video degradation at the signal level in audiovisual speaker identification (AVSI). The VSR estimation is formulated using a Gaussian fuzzy membership function (GFMF) to measure lighting variations. The variance parameters of GFMF are optimized in order to maximize the performance of the overall AVSI. The experimental results show that the proposed method outperforms the score-based reliability measuring technique. key words: visual signal reliability measure, Gaussian fuzzy membership function, audio-visual speaker identification

show abstract

Section: Baseline Avsi System: Score-based Fusionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Visual Signal Reliability for Robust Audio-Visual Speaker Identification

Tariquzzaman

Kim

et al. 2011

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…Moreover, speaker identification by using lips information is performed in (13) (14) . If these two technologies are collaborated, a human surveillance system with a mobile robot can be achieved, as mentioned in (15) .…”

Section: Introductionmentioning

confidence: 99%

High Speed Genetic Lips Detection by Dynamic Search Domain Control

et al. 2007

View full text Add to dashboard Cite

In this paper, high-speed size and orientation invariant lips detection of a talking person in an active scene using template matching and genetic algorithms is proposed (refer to Fig. 1). As part of the objectives, we also try to acquire numerical parameters to represent the lips. The information is very important for many applications, where high performance is required, such as audio-visual speech recognition, and personal mobile devices interfaces. The difficulty in lips detection is mainly due to deformations and geometric changes of the lips during speech and the active scene by free camera motion. In order to enhance the performance in speed and accuracy, initially, the performance is improved on a single still image, that is, the base of video processing.Our proposed system is based on template matching using genetic algorithms (GA). Only one template is prepared per experiment. The template is the closed mouth of a subject, because the application is for personal devices. In our previous study, the main problem was trade-off between search accuracy and search speed. To overcome this problem, we use two methods: scaling window and dynamic search domain control (SD-Control).The transition of the objective value on the proposed system and the no SD-Control are plotted in Fig. 2. At points from (a) to (d) in Figs. 2, the transition of GA exploration is exactly the same in both cases. The reason is that the same random numbers are used in both cases. Between points (d) and (e) in Fig. 2, the differences in the methods appear. This is because, the search domain is controlled from point (d) in Fig. 2. The elite individual of the no SD-Control do not change from point (c) to (g), hence it is concluded that the GA exploration gets trapped into a local optimum. The reason why the exploration was trapped into a local optimum is that the small population loses the population diversity and causes premature convergence.On the other hand, in case of the proposed system, the evolution make good progress from generation = 117 (point from (e) to (i)). This indicates that the GA of the proposed system can escape from the local maximum. The reason for this is that a search domain is reduced by the dynamic SD-Control and the GA can explore the reduced search domain with meticulous detail. This means that the GA in the proposed system performs not only global optimization but also local optimization. The result of our demonstration clearly shows that a trade-off between exploration accuracy and speed is overcome by the dynamic SD-Control.We achieved a lips detection accuracy of 91.33% at an average processing time of 33.70 milliseconds per frame. Stephen Karungaru * * Non-member Minoru Fukumi * * MemberIn this paper, high-speed size and orientation invariant lips detection of a talking person in an active scene using template matching and genetic algorithms is proposed. As part of the objectives, we also try to acquire numerical parameters to represent the lips. The information is very important for many applications, where ...

show abstract

“…One popular approach is to fuse the scores obtained from modality-specific classifiers. For example, in [1][2] the scores from a lip recognizer are fused with those from a speaker recognizer, and in [3] a face classifier is combined with a voice classifier using a variety of combination rules. These types of systems, however, require multiple sensors, which tend to increase system costs and require extra cooperation from users, e.g.…”

Section: Introductionmentioning

confidence: 99%

Probabilistic Fusion of Sorted Score Sequences for Robust Speaker Verification

Cheung

Mak

Kung

2005

Intelligent Multimedia Processing With Soft Computing

View full text Add to dashboard Cite

Abstract. Fusion techniques have been widely used in multi-modal biometric authentication systems. While these techniques are mainly applied to combine the outputs of modality-dependent classifiers, they can also be applied to fuse the decisions or scores from a single modality. The idea is to consider the multiple samples extracted from a single modality as independent but coming from the same source. In this chapter, we propose a single-source, multi-sample data-dependent fusion algorithm for speaker verification. The algorithm is data-dependent in that the fusion weights are dependent on the verification scores and the prior score statistics of claimed speakers and background speakers. To obtain the best out of the speaker's scores, scores from multiple utterances are sorted before they are probabilistically combined. Evaluations based on 150 speakers from a GSM-transcoded corpus are presented. Results show that data-dependent fusion of speaker's scores is significantly better than the conventional score averaging approach. It was also found that the proposed fusion algorithm can be further enhanced by sorting the score sequences before they are probabilistically combined.

show abstract

Adaptive Fusion of Speech and Lip Information for Robust Speaker Identification

Cited by 48 publications

References 28 publications

A Visual Signal Reliability for Robust Audio-Visual Speaker Identification

A Visual Signal Reliability for Robust Audio-Visual Speaker Identification

High Speed Genetic Lips Detection by Dynamic Search Domain Control

Probabilistic Fusion of Sorted Score Sequences for Robust Speaker Verification

Contact Info

Product

Resources

About