The Machine Knows What You Are Hiding: An Automatic Micro-expression Recognition System

Wu, Qi; Shen, Xunbing; Fu, Xiaolan

doi:10.1007/978-3-642-24571-8_16

Cited by 73 publications

(39 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…They were formally named microexpressions by Ekman (Ekman & Friesen, 1969) and presently there is great interest in this subject in research projects related to lying and deception detection (Ekman, 2009;Matsumoto & Hwang, 2011;Porter & ten Brinke, 2008, 2010Vrij et al, 2010). There are also works using computer vision techniques and learning algorithms to automatically recognize microexpressions in video sequences (Metaxas & Zhang, 2013;Michael, Dilsizian, Metaxas, & Burgoon, 2010;Pfister, Li, Zhao, & Pietikainen, 2011;Shreve, Godavarthy, Goldgof, & Sarkar, 2011;Wu, Shen, & Fu, 2011). Recently databases have been created (Yan, Wu, Liu, Wang, & Fu, 2013;X.…”

Section: Related Workmentioning

confidence: 99%

Investigating Macroexpressions and Microexpressions in Computer Graphics Animated Faces

Queiroz

Musse

Badler

2014

Presence: Teleoperators and Virtual Environments

View full text Add to dashboard Cite

Due to varied personal, social, or even cultural situations, people sometimes conceal or mask their true emotions. These suppressed emotions can be expressed in a very subtle way by brief movements called microexpressions. We investigate human subjects' perception of hidden emotions in virtual faces, inspired by recent psychological experiments. We created animations with virtual faces showing some facial expressions and inserted brief secondary expressions in some sequences, in order to try to convey a subtle second emotion in the character. Our evaluation methodology consists of two sets of experiments, with three different sets of questions. The first experiment verifies that the accuracy and concordance of the participant's responses with synthetic faces matches the empirical results done with photos of real people in the paper by X.-b. Shen, Q. Wu, and X.-l. Fu, 2012, "Effects of the duration of expressions on the recognition of microexpressions," Journal of Zhejiang University Science B, 13(3), 221-230. The second experiment verifies whether participants could perceive and identify primary and secondary emotions in virtual faces. The third experiment tries to evaluate the participant's perception of realism, deceit, and valence of the emotions. Our results show that most of the participants recognized the foreground (macro) emotion and most of the time they perceived the presence of the second (micro) emotion in the animations, although they did not identify it correctly in some samples. This experiment exposes the benefits of conveying microexpressions in computer graphics characters, as they may visually enhance a character's emotional depth through subliminal microexpression cues, and consequently increase the perceived social complexity and believability (Shen, Wu, & Fu, 2012). The second experiment verifies if participants could perceive and identify primary and secondary emotions in virtual faces. The third tries to evaluate the participant's perception of realism, deceit and valence of the emotions. Our results show that most of the participants recognized the foreground (macro) emotion and most of the time they had perceived the presence of the second (micro) emotion in the animations, although they did not identify it correctly in some samples. This experiment exposes the benefits of conveying microexpressions in Computer Graphics characters, as they may visually enhance a character's emotional "depth" through subliminal microexpression cues and consequently increase its perceived social complexity and believability.

show abstract

Section: Related Workmentioning

confidence: 99%

Investigating Macroexpressions and Microexpressions in Computer Graphics Animated Faces

Queiroz

Musse

Badler

2014

Presence: Teleoperators and Virtual Environments

View full text Add to dashboard Cite

show abstract

“…In this part, we review some existing microexpression analysing approaches as well as the deep learning techniques. Most work on micro-expressions in the field of Computer Vision has mainly focused on micro-expression recognition [10,19,22,23,28,29]. Micro-expression recognition task is defined as recognising the emotional label of well-segmented video containing micro-expression from start to end.…”

Section: Related Workmentioning

confidence: 99%

“…The reported results were carried out within each face subregions. Wu et al [28] extracted features using Gabor filters and used Support Vector Machine (SVM) to recognise them. Polikovsky et al [23] proposed to utilise Active Shape Model (ASM) model to detect facial landmarks which used to segment face area into twelve subregions.…”

Section: Related Workmentioning

confidence: 99%

Image Based Facial Micro-Expression Recognition Using Deep Learning on Small Datasets

Takalkar

2017

2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA)

View full text Add to dashboard Cite

Abstract-Facial micro-expression refers to split-second muscle changes in the face, indicating that a person is either consciously or unconsciously suppressing their true emotions and even mental health. Therefore, micro-expression recognition attracts increasing research efforts in both fields of psychology and computer vision. Existing research on micro-expression recognition has mainly used hand-crafted features, for example, Local Binary Pattern-Three Orthogonal Planes (LBP-TOP), Gabor filter and optical flow. Recently, Deep Convolutional neural systems have demonstrated a high degree effectiveness for difficult face recognition tasks. This paper explores the possible use of deep learning for micro-expression recognition. To develop a reliable deep neural network extensive training sets are required with a huge number of labeled image samples. However, micro-expression recognition is a challenging task due to the repressed facial appearance and short duration, which results in the lack of training data. In this paper, we propose to generate extensive training datasets of synthetic images using data augmentation on CASME and CASME II databases. Then, these datasets are combined to tune a satisfactory CNN-based microexpression recognizer. Experimental results demonstrate the effectiveness of the proposed CNN approach in image based micro-expression recognition and present comparable results with the best-related works.

show abstract

“…Capturing microexpressions in video has recently received much attention in computer vision and affective computing [47,32,39], and a few microexpression databases have been made available [32,21,48]. Wu et al [47] applied the Gabor filter and GentleSVM to classify microexpressions, while Pfister et al [32] proposed a method to temporally interpolate facial features using graph embedding.…”

Section: Related Workmentioning

confidence: 99%

“…Wu et al [47] applied the Gabor filter and GentleSVM to classify microexpressions, while Pfister et al [32] proposed a method to temporally interpolate facial features using graph embedding. Shreve et al [39] used facial strain patterns to detect both macro-and microexpressions.…”

Section: Related Workmentioning

confidence: 99%

Learning a sparse codebook of facial and body microexpressions for emotion recognition

Song¹,

Morency²,

Davis³

2013

Proceedings of the 15th ACM on International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Obtaining a compact and discriminative representation of facial and body expressions is a difficult problem in emotion recognition. Part of the difficulty is capturing microexpressions, i.e., short, involuntary expressions that last for only a fraction of a second: at a micro-temporal scale, there are so many other subtle face and body movements that do not convey semantically meaningful information. We present a novel approach to this problem by exploiting the sparsity of the frequent micro-temporal motion patterns. Local space-time features are extracted over the face and body region for a very short time period, e.g., few milliseconds. A codebook of microexpressions is learned from the data and used to encode the features in a sparse manner. This allows us to obtain a representation that captures the most salient motion patterns of the face and body at a micro-temporal scale. Experiments performed on the AVEC 2012 dataset show our approach achieving the best published performance on the expectation dimension based solely on visual features. We also report experimental results on audio-visual emotion recognition, comparing early and late data fusion techniques.

show abstract

The Machine Knows What You Are Hiding: An Automatic Micro-expression Recognition System

Cited by 73 publications

References 21 publications

Investigating Macroexpressions and Microexpressions in Computer Graphics Animated Faces

Investigating Macroexpressions and Microexpressions in Computer Graphics Animated Faces

Image Based Facial Micro-Expression Recognition Using Deep Learning on Small Datasets

Learning a sparse codebook of facial and body microexpressions for emotion recognition

Contact Info

Product

Resources

About