Audio-Visual Speech Recognition Using MPEG-4 Compliant Visual Features

Abstract

The most important challenges in AVSR and the focus of most research are the features that are extracted, and when combined give better results. The other challenge is the resulted feature here of nature are large in size, then prefers here to reduce the features by use of an appropriate way to reduce these data with ensure have their properties after downsizing. The System that is presented in this research is for recognition a group of Arabic words voices, from one to ten words. In the acoustic parts the features were extracted of coefficients MFCC, LPC,FFT to be determine which type of these features is efficient in AVSR .All these types of feature are showed efficient results but MFCC is the best. The visual features are calculated of DCT matrix, and the features are extracted by applying the zigzag scan. In the reduction features stage, several methods of data reducing have been implemented; they are LDA, PCA and SVD. Each method are applied to the data separately. The KNN models are used in the stage of recognition, where the testing is implemented on dependent and independent database of words from one to ten. The final results that obtained are efficient and encouraging.

Discriminative Analysis of Lip Motion Features for Speaker Identification and Speech-Reading

“…There exist many techniques in the literature that attempt to solve the lip segmentation/tracking problem [12], [29]- [35]. The performance of these techniques usually depend on acquisition specifics such as image quality, resolution, head pose and illumination conditions.…”

“…Deformable templates [4], [5], active shape models (ASM) [6], [10], [11], and snakes [12] have been used to obtain different lip geometry features; however, they all suffer from complex feature extraction and training procedures. In [5], Gaussian mixture models (GMM) are used to model both the lip and the non-lip region, and lip tracking is performed by deformable templates.…”

“…Aleksic et al [12] use gradient vector flow (GVF) snakes to extract outer lip contour and calculate the lip movement at ten predefined points by point-wise coordinate difference. They then reduce the feature dimension by PCA and use lip features together with other facial animation features.…”

Parametric models for facial features segmentation

“…Though the discriminant analysis is followed by a smoothing step, the segmentation remains noisy. Snakes [16] have been widely used for lip segmentation ( [1] [20]) because snakes can take into account in a same framework smoothing and elasticity constraints. Snake-based methods yield to interesting results but the main drawback is the tuning of several parameters.…”

Gestural Interfaces for Hearing-Impaired Communication

“…Significant research has been carried out to accurately obtain the outer lip contour. One of the most popular approaches is using snakes (Kass et al 1988), which have the ability to take smoothing and elasticity constraints into account (Terzopoulos and Waters 1993;Aleksic et al 2002). Another popular approach is using active shape models and appearance shape models.…”