Performance Analysis of Lip Synchronization Using LPC, MFCC and PLP Speech Parameters

Abstract

Many multimedia applications and entertainment industry products like games, cartoons and film dubbing require speech driven face animation and audio-video synchronization. Only Automatic Speech Recognition system (ASR) does not give good results in noisy environment. Audio Visual Speech Recognition system plays vital role in such harsh environment as it uses both -audio and visual -information. In this paper, we have proposed a novel approach with enhanced performance over traditional methods that have been reported so far. Our algorithm works on the bases of acoustic and visual parameters to achieve better results. We have tested our system for English language using LPC, MFCC and PLP parameters of the speech. Lip parameters like lip width, lip height etc are extracted from the video and these both acoustic and visual parameters are used to train systems like Artificial Neural Network (ANN), Vector Quantization (VQ), Dynamic Time Warping (DTW), Support Vector Machine (SVM). We have employed neural network in our research work with LPC, MFCC and PLP parameters. Results show that our system is giving very good response against tested vowels.

Chain Code and Holistic Features Based OCR System for Printed Devanagari Script Using ANN and SVM

“…We have used three layer feed forward back propagation neural network (FFBPNN) with input, hidden and output layer. Two layers FFBP is perhaps the best choice for classification [30]. In our experiment, for binary features, we have used 9, 6 and 1 neurons in input, hidden and output layer respectively.…”

A Systematic Literature Review: Facial Expression and Lip Movement Synchronization of an Audio Track

“…In Automatic Dubbing, lip movement synchronization refers to aligning and synchronizing the lip movements of a character with the corresponding dubbed audio track [1]. The primary objective of lip movement synchronization is to ensure a seamless lip-syncing effect by closely matching the lip movements of the character with the spoken words [2], [3].…”

Extraction of Lip features for the Identification of Vowels Utterances using MFCC and Geometrical Aspects

“…Mel Frequency Cepstral Coefficients (MFCC) are quiet well applied features in most of the speech identification applications as well as speakers. In 1980s, considerable attempts has been done to develop MFCC [3], [4]. While employing MFCC in the development of applications, many problems such as use of frequency estimation algorithms, design of the efficient filter banks and the total selected features in extracting the speech and its dynamics all play a significant role in the performance of speech recognition systems.…”