Audio Visual Speech Recognition and Segmentation Based on DBN Models

Jiang, Dongmei; Lv, Guoyun; Ravyse, Ilse; Jiang, Xiaoyue; Zhang, Yanning; Sahli, Hichem; Zhao, Ran

doi:10.5772/4748

Cited by 1 publication

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Average optical flows of six regions located among these ten points are calculated. These six regions are bounded among the following points: a) 1,3,5,7 b) 2,4,5,7 c) 3,6,7 d) 4,6,7 e) 3,6,8,10 f) 4,6,910 These areas which can be seen in Figure 2 part (a) are chosen due to the fact that each of them has a different movement compared to the other parts while speaking. And it was expected that the optical flow vector illustrates the correct direction in each of them.…”

Section: ) Right Nose Hole 2) Left Nose Hole 3) Right Corner Of Lip mentioning

confidence: 99%

See 1 more Smart Citation

Automatic Speech Segmentation Based On Audio and Optical Flow Visual Classification

Torabi

Nilchi

2014

IJIGSP

View full text Add to dashboard Cite

Automatic speech segmentation as an important part of speech recognition system (ASR) is highly noise dependent. Noise is made by changes in the communication channel, background, level of speaking etc. In recent years, many researchers have proposed noise cancelation techniques and have added visual features from speaker's face to reduce the effect of noise on ASR systems. Removing noise from audio signals depends on the type of the noise; so it cannot be used as a general solution. Adding visual features improve this lack of efficiency, but advanced methods of this type need manual extraction of visual features. In this paper we propose a completely automatic system which uses optical flow vectors from speaker's image sequence to obtain visual features. Then, Hidden Markov Models are trained to segment audio signals from image sequences and audio features based on extracted optical flow. The developed segmentation system based on such method acts totally automatic and become more robust to noise.

show abstract

Section: ) Right Nose Hole 2) Left Nose Hole 3) Right Corner Of Lip mentioning

confidence: 99%

“…They extracted those features using some blue dots which were attached to speaker lips manually while recording database. Dongmei Jiang et al [7] used some extracted geometric features from speaker lips too. They specified many points on lip contours manually and induced their features by calculating the distance between these points.…”

Section: Introductionmentioning

confidence: 99%

Automatic Speech Segmentation Based On Audio and Optical Flow Visual Classification

Torabi

Nilchi

2014

IJIGSP

View full text Add to dashboard Cite

show abstract

Audio Visual Speech Recognition and Segmentation Based on DBN Models

Cited by 1 publication

References 11 publications

Automatic Speech Segmentation Based On Audio and Optical Flow Visual Classification

Automatic Speech Segmentation Based On Audio and Optical Flow Visual Classification

Contact Info

Product

Resources

About