Model-Based Global and Local Motion Estimation for Videoconference Sequences

Abstract

In this work, we present an algorithm for face 3-D motion estimation in videoconference sequences. The algorithm is able to estimate both the position of the face as an object in 3-D space (global motion) and the movements of portions of the face, like the mouth or the eyebrows (local motion). The algorithm uses a modified version of the standard 3-D face model CANDIDE. We present various techniques to increase robustness of the global motion estimation which is based on feature tracking and an extended Kalman filter. Global motion estimation is used as a starting point for local motion detection in the mouth and eyebrow areas. To this purpose, synthetic images of these areas (templates) are generated with texture mapping techniques, and then compared to the corresponding regions in the current frame. A set of parameters, called action unit vectors (AUVs) influences the shape of the synthetic mouth and eyebrows. The optimal AUV values are determined via a gradient-based minimization procedure of the error energy between the templates and the actual face areas. The proposed scheme is robust and was tested with success on sequences of many hundreds of frames.Index Terms-Adaptive Kalman filtering, block matching, eyebrow tracking, face tracking, feature tracking, lip tracking, modelbased coding, template matching.

A Real-Time Technique for Spatio–Temporal Video Noise Estimation

“…Since the proposed algorithm is designed to be context-free, there are no restrictions on the original signal . The division of into cubes with spatial indexes and and temporal index is done using (2) where is a set of indexes (locations) making a cube of size centered around the 3-D point and is the set of pixel values at those locations. A single spatial plain in is denoted which can be isolated using…”

“…1(c) followed by masks in Fig. 2 to the video signal cubes [see (2)]. We select the most homogeneous cubes only based on each for noise variance estimation.…”

“…For example, in segmentation [1], the binarization process can be adapted to the noise level for more stable object masks. Also, accurate information about the noise level is vital to robust motion estimation [2] to tune the outliers rejection phase for more reliable motion vectors or parameters.…”

Superpixel-based image noise variance estimation with local statistical assessment

“…The accuracy of many algorithms significantly relies on well hand-tuned parameter adjustments to account for variations in noise [1][2][3]. To automate the process and achieve reliable procedures, the capability for accurate noise estimation is essential to motion estimation, edge detection, super-resolution, restoration, shape-from-shading, feature extraction, and object recognition [4][5][6][7][8][9]. In particular, image noise having a Gaussian-like distribution is quite often encountered, and it is characterized by adding to each pixel a random value obtained from a zero-mean Gaussian distribution, whose variance determines the magnitude of the corrupting noise.…”