A new optimization procedure for extracting the point-based lip contour using active shape model

Sum, K.L.; Lau, Wing Cheong; Leung, S.H.; Liew, Alan Wee-Chung; Tse, Kai-Ming

doi:10.1109/icassp.2001.941212

Cited by 27 publications

(16 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Experimental results have demonstrated that our lip extraction algorithms [4], [5] perform well without the need of stringent lighting conditions. However, a major concern is the extraction of the inner mouth features which can be affected by the pose and lighting condition.…”

Section: Discussionmentioning

confidence: 90%

“…In order to remove the variations caused by varying camera distance, they are normalized with respect to the values of the first image in the image sequence and are denoted as . Second, as the ASM lip model is adopted to describe the lip contour, the shape variation is contained in the eigenvalues [5]. Since the first few eigenvectors corresponding to the largest eigenvalues are sufficient for modeling the shape variation, only the first three weight values denoted by are incorporated into the visual features vector.…”

Section: Visual Feature Extractionmentioning

confidence: 99%

“…Various methods have been proposed for lip segmentation and modeling [2]- [5]. Edge-based [2] and Markov random field (MRF) based [3] approaches are the two widely used techniques.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Automatic Lipreading System for Spoken Digits With Limited Training Data

Wang

Liew

Lau

et al. 2008

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

It is well known that visual cues of lip movement contain important speech relevant information. This paper presents an automatic lipreading system for small vocabulary speech recognition tasks. Using the lip segmentation and modeling techniques we developed earlier, we obtain a visual feature vector composed of outer and inner mouth features from the lip image sequence for recognition. A spline representation is employed to transform the discrete-time sampled features from the video frames into the continuous domain. The spline coefficients in the same word class are constrained to have similar expression and are estimated from the training data by the EM algorithm. For the multiple-speaker/speaker-independent recognition task, an adaptive multimodel approach is proposed to handle the variations caused by various talking styles. After building the appropriate word models from the spline coefficients, a maximum likelihood classification approach is taken for the recognition. Lip image sequences of English digits from 0 to 9 have been collected for the recognition test. Two widely used classification methods, HMM and RDA, have been adopted for comparison and the results demonstrate that the proposed algorithm deliver the best performance among these methods.Index Terms-Lipreading, visual feature extraction, visual speech recognition.

show abstract

Section: Discussionmentioning

confidence: 90%

Section: Visual Feature Extractionmentioning

confidence: 99%

See 1 more Smart Citation

An Automatic Lipreading System for Spoken Digits With Limited Training Data

Wang

Liew

Lau

et al. 2008

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The detection of these points is referred to as face alignment. The Active Appearance Model (AAM) [1] and Active Shape Model (ASM) [17] are well-known face alignment models. In this paper, we employed a Constrained Local Model (CLM) [2], [15].…”

Section: Related Workmentioning

confidence: 99%

Audio-Visual Speech Recognition Using Convolutive Bottleneck Networks for a Person with Severe Hearing Loss

Takashima

Kakihara

Aihara

et al. 2015

IPSJ Transactions on Computer Vision and Applications

View full text Add to dashboard Cite

In this paper, we propose an audio-visual speech recognition system for a person with an articulation disorder resulting from severe hearing loss. In the case of a person with this type of articulation disorder, the speech style is quite different from with the result that of people without hearing loss that a speaker-independent model for unimpaired persons is hardly useful for recognizing it. We investigate in this paper an audio-visual speech recognition system for a person with severe hearing loss in noisy environments, where a robust feature extraction method using a convolutive bottleneck network (CBN) is applied to audio-visual data. We confirmed the effectiveness of this approach through word-recognition experiments in noisy environments, where the CBN-based feature extraction method outperformed the conventional methods.

show abstract

“…In the proposed approach, Wang made use of the first three weight values as they are sufficient for modeling shape variation iii) Inner mouth features: teeth and mouth opening From the technique in [7], teeth region area is calculated and normalized against the entire mouth, which is then enclosed by the outer lip contour. Inner mouth area is yet another parameter that is taken into account, this is obtained by analyzing luminance of the teeth and teeth information along the central axis of the mouth.…”

Section: Figure 2: Appearance Of Lip During Various Soundmentioning

confidence: 99%

Implication and Utilization of various Lip Reading Techniques

Khan¹,

Azmi²,

Nair³

et al. 2017

IJCA

View full text Add to dashboard Cite

Ears are one of the most important sensory organs of the human body. It is one of our primary sensors along with our eyes, nose, skin etc. However, while many people may have perfectly functional ears some suffer from partial or total hearing loss or hearing impairment. In order to tackle this, many suggest the use of hearing aids, sign language, cochlear implants. But of all the different ways, one of the best way to beat this inability is to train oneself to lip read. In simple words, lip reading is the interpretation of the movements of not only the lips but also the face and tongue. While many people are excellent at lip reading, some find it a challenge too difficult to overcome. In regard to this, one cannot deny that whenever some human falls short of something, a manmade technology can make up for the shortcoming to a certain extent. As a result, over the years a lot of time and effort has been invested in studying various lip reading algorithms and speech automations. Through this research paper, the aim is to put forth a study on various lip-reading algorithms, their approaches, their innovations and the central ideas behind their proposed studies.

show abstract

A new optimization procedure for extracting the point-based lip contour using active shape model

Cited by 27 publications

References 6 publications

An Automatic Lipreading System for Spoken Digits With Limited Training Data

An Automatic Lipreading System for Spoken Digits With Limited Training Data

Audio-Visual Speech Recognition Using Convolutive Bottleneck Networks for a Person with Severe Hearing Loss

Implication and Utilization of various Lip Reading Techniques

Contact Info

Product

Resources

About