Micro-Expression Recognition Using Micro-Variation Boosted Heat Areas

Zhang, Mingyue; Huan, Zhaoxin; Shang, Lin

doi:10.1007/978-3-030-60639-8_44

Cited by 3 publications

(6 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Loss: Cross-entropy [118] Metric [119], [120] Margin [121], [122], [123] Imbalance [86], [124] Training strategy: Finetune [69], [82] Knowledge distillation [64], [113], [114] Domain adaption [115], [116], [117] Architecture Multi-task learning: landmark [112] Gender [71] AU [86] Multi-binary-class [104] Cascade: CNN+RNN [106] CNN+LSTM [100], [107], [108], [109], [110], [111] CNN+GCN [53] Multiple stream: Handcraft+CNN: Dual [103], [104], [105] Different blocks: Dual [98], [99], [100] Triplet [101], [102] Same block: Dual [90], [91], [92] Triple [93], [94], [95], [96] Four [97] Single stream: 2D [59], [69], [88] 3D [82],…”

Section: Networkmentioning

confidence: 99%

“…In addition, since MEs have specific muscular activations on the face, MEs are related with local regional changes [167]. Therefore, it is crucial to highlight the representation on RoIs [8], [108]. Several approaches [98], [168], [169], [170], [171], [172] have shown the benefit of enhancing spatial encoding with attention module.…”

Section: Network Blockmentioning

confidence: 99%

“…In addition, multiple MER works combined multistream and cascaded structure to further explore the multiview series information. VGGFace2+LSTM [109], Temporal Facial Micro-Variation Network (TFMVN) [108] and MER with Ternary Attentions (MERTA) [96] developed three stream VGGNets followed by LSTMs to extract multiview spatio-temporal features. Different from above works, Khor et al [107] proposed an Enriched Long-term Recurrent Convolutional Network (ELRCN) adding one VGG+LSTM path with channel-wise stacking inputs for spatial enrichment.…”

Section: Cascaded Networkmentioning

confidence: 99%

See 2 more Smart Citations

Deep Learning for Micro-Expression Recognition: A Survey

Wei

Liu

et al. 2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Micro-expressions (MEs) are involuntary facial movements revealing people's hidden feelings in high-stake situations and have practical importance in various fields. Early methods for Micro-expression Recognition (MER) are mainly based on traditional features. Recently, with the success of Deep Learning (DL) in various tasks, neural networks have received increasing interest in MER. Different from macro-expressions, MEs are spontaneous, subtle, and rapid facial movements, leading to difficult data collection and annotation, thus publicly available datasets are usually small-scale. Currently, various DL approaches have been proposed to solve the ME issues and improve MER performance. In this survey, we provide a comprehensive review of deep MER and define a new taxonomy for the field encompassing all aspects of MER based on DL, including datasets, each step of the deep MER pipeline, and performance comparisons of the most influential methods. The basic approaches and advanced developments are summarized and discussed for each aspect. Additionally, we conclude the remaining challenges and potential directions for the design of robust MER systems. Finally, ethical considerations in MER are discussed. To the best of our knowledge, this is the first survey of deep MER methods, and this survey can serve as a reference point for future MER research.

show abstract

Section: Networkmentioning

confidence: 99%

Section: Network Blockmentioning

confidence: 99%

Section: Cascaded Networkmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning for Micro-Expression Recognition: A Survey

Wei

Liu

et al. 2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

show abstract

“…Since MEs consist of specific muscular activations on the face and have low intensity, MEs are related with local regional changes [183]. Therefore, it is crucial to highlight the representation on regions of interest (RoIs) [8], [113]. Several approaches [103], [184]- [186] have shown the benefit of enhancing spatial encoding with attention module.…”

Section: A Network Blockmentioning

confidence: 99%

“…[115] and [117] combined VGG-FACE and 3DCNN with LSTMs in series, respectively. Temporal Facial Micro-Variation Network (TFMVN) [113] and MERTA [114] developed three stream VGGNets followed by LSTMs to extract multi-view spatio-temporal features for MER. Specifically, TFMVN employed three facial regions (eye, nose and mouth) as inputs and heatmap served as structure cue to guide micro-exression feature learning.…”

Section: B Network Structurementioning

confidence: 99%

Deep Learning for Micro-expression Recognition: A Survey

Li¹,

Wei²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Micro-expressions (MEs) are involuntary facial movements revealing people's hidden feelings in high-stake situations and have practical importance in medical treatment, national security, interrogations and many human-computer interaction systems. Early methods for MER mainly based on traditional appearance and geometry features. Recently, with the success of deep learning (DL) in various fields, neural networks have received increasing interests in MER. Different from macro-expressions, MEs are spontaneous, subtle, and rapid facial movements, leading to difficult data collection, thus have small-scale datasets. DL based MER becomes challenging due to above ME characters. To data, various DL approaches have been proposed to solve the ME issues and improve MER performance. In this survey, we provide a comprehensive review of deep microexpression recognition (MER), including datasets, deep MER pipeline, and the bench-marking of most influential methods. This survey defines a new taxonomy for the field, encompassing all aspects of MER based on DL. For each aspect, the basic approaches and advanced developments are summarized and discussed. In addition, we conclude the remaining challenges and and potential directions for the design of robust deep MER systems. To the best of our knowledge, this is the first survey of deep MER methods, and this survey can serve as a reference point for future MER research.

show abstract

4DME: A Spontaneous 4D Micro-Expression Dataset With Multimodalities

Cheng

et al. 2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Micro-expressions (ME) are a special form of facial expressions which may occur when people try to hide their true feelings for some reasons. MEs are important clues to reveal people's true feelings, but are difficult or impossible to be captured by ordinary persons with naked-eyes as they are very short and subtle. It is expected that robust computer vision methods can be developed to automatically analyze MEs which requires lots of ME data. The current ME datasets are insufficient, and mostly contain only one single form of 2D color videos. Researches on 4D data of ordinary facial expressions have prospered, but so far no 4D data is available in ME study. In the current study, we introduce the 4DME dataset: a new spontaneous ME dataset which includes 4D data along with three other video modalities. Both micro-and macro-expression clips are labeled out in 4DME, and 22 AU labels and five categories of emotion labels are annotated. Experiments are carried out using three 2D-based methods and one 4D-based method to provide baseline results. The results indicate that the 4D data can potentially benefit ME recognition. The 4DME dataset could be used for developing 4D-based approaches, or exploring fusion of multiple video sources (e.g., texture and depth) for the task of ME analysis in future. Besides, we also emphasize the importance of forming a clear and unified criteria of ME annotation for future ME data collection studies. Several key questions related with ME annotation are listed and discussed in depth, especially about the relationship between AUs and ME emotion categories. A preliminary AU-Emo mapping table is proposed with justified explanations and supportive experimental results. Several unsolved issues are also summarized for future work.

show abstract

Micro-Expression Recognition Using Micro-Variation Boosted Heat Areas

Cited by 3 publications

References 21 publications

Deep Learning for Micro-Expression Recognition: A Survey

Deep Learning for Micro-Expression Recognition: A Survey

Deep Learning for Micro-expression Recognition: A Survey

4DME: A Spontaneous 4D Micro-Expression Dataset With Multimodalities

Contact Info

Product

Resources

About