Kernel-Based Lip Shape Clustering with Phoneme Recognition for Real-Time Voice Driven Talking Face

Shih, Po-Yi; Wang, Jhing-Fa; Chen, Zongyou

doi:10.1007/978-3-642-13318-3_64

Cited by 1 publication

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conveying speech alongside gesture or expression is difficult, the question however is how can a realistic virtual human that incorporates emotions as well as gaze and speech behaviors be achieved. Describe a real time voice driven method using which a speaker's lip shape is synchronized with the corresponding speech signal for a low bandwidth mobile devices using sum of absolute difference (SAD) as vowel lip shape likelihood to cluster into categories then adjust the source and destination pictures of lip shape in the transparent level using alpha blending (Shih et al, 2010). The physics-based method uses the laws of physics and muscle forces to drive the motion of the face.…”

Section: Visual Motion Speech Synthesismentioning

confidence: 99%

See 1 more Smart Citation

Realistic Lip Syncing for Virtual Character Using Common Viseme Set

Ali¹,

Sulong²,

Kolivand³

2015

CIS

View full text Add to dashboard Cite

Speech is one of the most important interaction methods between the humans. Therefore, most of avatar researches focus on this area with significant attention. Creating animated speech requires a facial model capable of representing the myriad shapes the human face expressions during speech. Moreover, a method to produce the correct shape at the correct time is also in order. One of the main challenges is to create precise lip movements of the avatar and synchronize it with a recorded audio. This paper proposes a new lip synchronization algorithm for realistic applications, which can be employed to generate synchronized facial movements among the audio generated from natural speech or through a text-to-speech engine. This method requires an animator to construct animations using a canonical set of visemes for all pair wise combination of a reduced phoneme set. These animations are then stitched together smoothly to construct the final animation.

show abstract

Section: Visual Motion Speech Synthesismentioning

confidence: 99%

“…With the development of communication systems, it is now possible for people to interact directly with real time video devices. Such a communication system allows synchronization of the lip shape character with the corresponding speech in a real time voice driven mobile device designed by Shih et al (2010).…”

Section: Expressive Speech Synthesismentioning

confidence: 99%