Hemiface mirroring: a new approach to reducing bandwidth requirements of audiovisual telecommunication

Abstract

Research has shown that the sight of a speaker's face can dramatically improve the intelligibility of speech. The addition of a video stream to telecommunications systems is, therefore, a highly desirable goal. Extending the content-based coding paradigm of MPEG-4, we present a novel method of reducing the amount of data required to represent video images of human speakers. This method is based around the mirroring of vertical halves of the speaker's face. We report an experiment that compared video of synthesized speaking faces (constructed by mirroring hemifaces) with that of "normal" (un-manipulated) speaking faces. The results of the experiment show that the synthesized faces provided the same benefits to speech perception as the normal faces. Development of this technique as an image-coding technique for audiovisual telecommunication applications is discussed.