Coding, analysis, interpretation, and recognition of facial expressions

Essa, Irfan; Pentland, Alex

doi:10.1109/34.598232

Cited by 755 publications

(395 citation statements)

References 24 publications

Supporting

Mentioning

378

Contrasting

Unclassified

Order By: Relevance

“…Rigid and deformable tracking of faces and facial features have been a very popular topic of research over the past twenty years (Black and Yacoob 1995;Lanitis et al 1995;Sobottka and Pitas 1996;Essa et al 1996Essa et al , 1997Oliver et al 1997;Decarlo and Metaxas 2000;Jepson et al 2003;Xiao et al 2004;Patras and Pantic 2004;Kim et al 2008;Ross et al 2008;Papandreou and Maragos 2008;Amberg et al 2009;Kalal et al 2010a;Koelstra et al 2010;Tresadern et al 2012;Tzimiropoulos and Pantic 2013;Xiong and De la Torre 2013;Liwicki et al 2013;Smeulders et al 2014;Asthana et al 2014;Li et al 2016a;Xiong and De la Torre 2015;Snape et al 2015;Tzimiropoulos 2015). In this section we provide an overview of face tracking spanning over the past twenty years up to the present day.…”

Section: Related Workmentioning

confidence: 99%

“…Deformable tracking of faces can be further subdivided into i) tracking of certain facial landmarks (Lanitis et al 1995;Black and Yacoob 1995;Sobottka and Pitas 1996;Xiao et al 2004;Patras and Pantic 2004;Papandreou and Maragos 2008;Amberg et al 2009;Tresadern et al 2012;Xiong and De la Torre 2013;Asthana et al 2014;Xiong and De la Torre 2015) or ii) tracking/estimation of dense facial motion Yacoob and Davis 1996;Essa et al 1997;Decarlo and Metaxas 2000;Koelstra et al 2010;Snape et al 2015). The latter category of estimating a dense facial motion through a model-based system was proposed by MIT Media lab in mid 1990's (Essa et al 1997Basu et al 1996). In particular, the method by tracks facial motion using optical flow computation coupled with a geometric and a physical (muscle) model describing the facial structure.…”

Section: Prior Artmentioning

confidence: 99%

See 1 more Smart Citation

A Comprehensive Performance Evaluation of Deformable Face Tracking “In-the-Wild”

et al. 2017

View full text Add to dashboard Cite

Recently, technologies such as face detection, facial landmark localisation and face recognition and verification have matured enough to provide effective and efficient solutions for imagery captured under arbitrary conditions (referred to as "in-the-wild"). This is partially attributed to the fact that comprehensive "in-the-wild" benchmarks have been developed for face detection, landmark localisation and recognition/verification. A very important technology that has not been thoroughly evaluated yet is deformable face tracking "in-the-wild". Until now, the performance has mainly been assessed qualitatively by visually assessing the result of a deformable face tracking technology on short videos. In this paper, we perform the first, to the best of our knowledge, thorough evaluation of state-of-theart deformable face tracking pipelines using the recently introduced 300 VW benchmark. We evaluate many different architectures focusing mainly on the task of on-line deformable face tracking. In particular, we compare the following general strategies: (a) generic face detection plus generic facial landmark localisation, (b) generic model free tracking plus generic facial landmark localisation, as well as (c) hybrid approaches using state-of-the-art face detection, model free tracking and facial landmark localisation technologies. Our evaluation reveals future avenues for further research on the topic.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Prior Artmentioning

confidence: 99%

A Comprehensive Performance Evaluation of Deformable Face Tracking “In-the-Wild”

et al. 2017

View full text Add to dashboard Cite

show abstract

“…1). In facial expression analysis [67], most systems extract either geometric-based [17] or appearance-based facial features, e.g. Gabor wavelets or local binary patterns [60], to recognise prototypic emotional expressions, such as anger or fear.…”

Section: Intent Profilingmentioning

confidence: 99%

Security and Surveillance

Gong

Loy

Xiang

2011

Visual Analysis of Humans

View full text Add to dashboard Cite

Human eyes are highly efficient devices for scanning through a large quantity of low-level visual sensory data and delivering selective information to one's brain for high-level semantic interpretation and gaining situational awareness. Over the last few decades, the computer vision community has endeavoured to bring about similar perceptual capabilities to artificial visual sensors. Substantial efforts have been made towards understanding static images of individual objects and the corresponding processes in the human visual system. This endeavour is intensified further by the need for understanding a massive quantity of video data, with the aim to comprehend multiple entities not only within a single image but also over time across multiple video frames for understanding their spatio-temporal relations. A significant application of video analysis and understanding is intelligent surveillance, which aims to interpret automatically human activity and detect unusual events that could pose a threat to public security and safety.

show abstract

“…One of the problems of fitting a 3D model to a single image is that depth is almost impossible to determine, and the tracking problem is generally underconstrained. As a result, 3D model-based nonrigid motion tracking techniques require either strong prior assumptions about the shape of the object (e.g., symmetry [57], deformable superquadrics [37,40,66], human faces [15,17,56]) or that multiple views of the object are available. By using a 2D deformable image template, the active blobs formulation has the advantage of being able to track general nonrigid motion within the image plane without the need to define a full 3D model nor strong prior assumptions about the shape of the object to be tracked.…”

Section: Related Workmentioning

confidence: 99%

Active blobs: region-based, deformable appearance models

Sclaroff

Isidoro

2003

Computer Vision and Image Understanding

View full text Add to dashboard Cite

A region-based approach to nonrigid motion tracking is described. Shape is defined in terms of a deformable triangular mesh that captures object shape plus a color texture map that captures object appearance. Photometric variations are also modeled. Nonrigid shape registration and motion tracking are achieved by posing the problem as an energy-based, robust minimization procedure. The approach provides robustness to occlusions, wrinkles, shadows, and specular highlights. The formulation is tailored to take advantage of texture mapping hardware available in many workstations, PCs, and game consoles. This enables nonrigid tracking at speeds approaching video rate. Ó 2003 Published by Elsevier Science (USA).

show abstract

Coding, analysis, interpretation, and recognition of facial expressions

Abstract: We describe a computer vision system for observing facial motion by using an optimal estimation optical flow method coupled with geometric, physical and motion-based dynamic models describing the facial structure.

Cited by 755 publications

References 24 publications

A Comprehensive Performance Evaluation of Deformable Face Tracking “In-the-Wild”

A Comprehensive Performance Evaluation of Deformable Face Tracking “In-the-Wild”

Security and Surveillance

Active blobs: region-based, deformable appearance models

Contact Info

Product

Resources

About