An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge

Xie, Hui; Lo, Ling; Shuai, Hong-Han; Cheng, Wen-Huang

doi:10.48550/arxiv.2012.11307

Cited by 5 publications

(11 citation statements)

References 168 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, to examine the proposed framework's efficiency, we have compared our results with other state-ofthe-art approaches. There are still some restriction to compare the proposed MicroNet performance with existing approaches as there is no standard evaluation protocol and class setting [64], [80]. Therefore, it is difficult to validate the proposed method by comparing with all existing MER approaches.…”

Section: A Implementation Detailsmentioning

confidence: 99%

A Motion Flow Guided MicroNet Framework for Micro Expression Recognition

Verma¹,

Vipparthi²,

Singh³

2022

Preprint

View full text Add to dashboard Cite

<p> The micro-expression recognition (MER) has gained high attention in real-world applications: human-computer interaction, depression estimation, virtual reality, etc. However, MER systems still face difficulty in capturing the subtle information with spatial appearance of micro expressions. In this paper, an efficient and robust motion flow-guided MicroNet framework is proposed for MER. The proposed MicroNet consists: motion flow generator (MFGen) and avalanche feature (AFeat) block. The MFGen is introduced to extract the temporal changes at expressive regions by analyzing the motion intensity of pixels among the frames. Further, the AFeat block is designed to capture the spatiotemporal feature using a multi-lateral complementary feature (MCFeat) block. The MCFeat block is proposed to elicit the coarse and deep edge responses from multi-scale receptive fields. Thus, the proposed MicroNet estimates not only momentary variations but also learns affective appearance features of micro expressions. Furthermore, the efficacy of the proposed MicroNet is examined by conducting experiments on two experimental setups using six comprehensive datasets: CASME-I, CASME-II, CAS(ME)2, SAMM, SMIC and COMPOSITE, with two validation schemes: leave-one-subject-out and cross-domain to demonstrate the generalization capacity and robustness of the proposed framework. Also, eight ablation experiments have been executed to validate each module’s role in the MicroNet framework.</p>

show abstract

Section: A Implementation Detailsmentioning

confidence: 99%

A Motion Flow Guided MicroNet Framework for Micro Expression Recognition

Verma¹,

Vipparthi²,

Singh³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The ordinary FER approaches and MER with traditional learning methods are not considered in this survey. Although a few MER surveys have discussed the historical evolution and algorithmic pipelines for MER [23], [24], [25], [26], [27], [28], they mainly focus on traditional methods and only introduce some recent DL approaches. The DL-based MER has not been discussed systematically and specifically.…”

Section: Introductionmentioning

confidence: 99%

Deep Learning for Micro-Expression Recognition: A Survey

Wei

Liu

et al. 2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Micro-expressions (MEs) are involuntary facial movements revealing people's hidden feelings in high-stake situations and have practical importance in various fields. Early methods for Micro-expression Recognition (MER) are mainly based on traditional features. Recently, with the success of Deep Learning (DL) in various tasks, neural networks have received increasing interest in MER. Different from macro-expressions, MEs are spontaneous, subtle, and rapid facial movements, leading to difficult data collection and annotation, thus publicly available datasets are usually small-scale. Currently, various DL approaches have been proposed to solve the ME issues and improve MER performance. In this survey, we provide a comprehensive review of deep MER and define a new taxonomy for the field encompassing all aspects of MER based on DL, including datasets, each step of the deep MER pipeline, and performance comparisons of the most influential methods. The basic approaches and advanced developments are summarized and discussed for each aspect. Additionally, we conclude the remaining challenges and potential directions for the design of robust MER systems. Finally, ethical considerations in MER are discussed. To the best of our knowledge, this is the first survey of deep MER methods, and this survey can serve as a reference point for future MER research.

show abstract

“…However, they have weak performance, when shaking. Nonetheless, these handcrafted methods depended on previous science and heuristics [21,27,28].…”

Section: Introductionmentioning

confidence: 99%

Spotting micro‐movements in image sequence by introducing intelligent cubic‐LBP

Esmaeili

Feghhi

Shahdi

2022

IET Image Processing

View full text Add to dashboard Cite

Invisible to naked human eyes, micro-movements are barely noticeable. There are wide ranges of micro-movement applications from spotting subtle changes in a volcano, vascular pulse, and blood vessel to micro-expression detection. In the latter, some evil thoughts can be unveiled, resulting in identifying crooks and lawbreakers and it arises from involuntary subtle and short-duration of facial muscles movements. Precise spotting of these tiny movements is possible only when multiple aspects of temporal images are scrutinized. Meanwhile, since motions often happen in one or two directions, it is rudimentary to extract complete feature sets in textural-based approaches such as cubic Local Binary Pattern (cubic-LBP). Approaches like cubic-LBP also have imposed an unnecessary computation burden. Hence, in this research, a novel method named intelligent cubic-LBP is proposed, which incorporates Convolutional Neural Network (CNN) model. This model learns to select the useful plane(s) automatically. Apex is then detected by applying Partial Autocorrelation Coefficient (PACF) on the selected plane(s). The experimental results show significant improvement in micro-movements identification. The accuracy of the apex frame detection has elevated to 10% and 17% in the Chinese Academy of Sciences Micro-Expressions (CASME) and the CASME II databases, respectively.

show abstract

An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge

Cited by 5 publications

References 168 publications

A Motion Flow Guided MicroNet Framework for Micro Expression Recognition

A Motion Flow Guided MicroNet Framework for Micro Expression Recognition

Deep Learning for Micro-Expression Recognition: A Survey

Spotting micro‐movements in image sequence by introducing intelligent cubic‐LBP

Contact Info

Product

Resources

About