Cross-database Micro Expression Recognition Based on Apex Frame Optical Flow and Multi-head Self-attention

Wen, Jiebin; Yang, Wenzhong; Wang, Lie-Jun; Wei, Wenyu; Tan, Sixiang; Yong-zhi, WU

doi:10.1007/978-981-16-0010-4_12

Cited by 4 publications

(4 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…7 (a)) achieves easy optimization and reduces the effect of the vanishing gradient problem. Multiple MER works [84], [99], [156], [157], [158] employed residual blocks for robust recognition on smallscale ME datasets. Instead of directly applying the shortcut connection, [159] further designed a convolutionable shortcut to learn the important residual information and AffectiveNet [160] introduced an MFL module learning the low-and high-level feature parallelly to increase the discriminative capability between the inter and intra-class variations.…”

Section: Network Blockmentioning

confidence: 99%

Deep Learning for Micro-Expression Recognition: A Survey

Wei

Liu

et al. 2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Micro-expressions (MEs) are involuntary facial movements revealing people's hidden feelings in high-stake situations and have practical importance in various fields. Early methods for Micro-expression Recognition (MER) are mainly based on traditional features. Recently, with the success of Deep Learning (DL) in various tasks, neural networks have received increasing interest in MER. Different from macro-expressions, MEs are spontaneous, subtle, and rapid facial movements, leading to difficult data collection and annotation, thus publicly available datasets are usually small-scale. Currently, various DL approaches have been proposed to solve the ME issues and improve MER performance. In this survey, we provide a comprehensive review of deep MER and define a new taxonomy for the field encompassing all aspects of MER based on DL, including datasets, each step of the deep MER pipeline, and performance comparisons of the most influential methods. The basic approaches and advanced developments are summarized and discussed for each aspect. Additionally, we conclude the remaining challenges and potential directions for the design of robust MER systems. Finally, ethical considerations in MER are discussed. To the best of our knowledge, this is the first survey of deep MER methods, and this survey can serve as a reference point for future MER research.

show abstract

Section: Network Blockmentioning

confidence: 99%

Deep Learning for Micro-Expression Recognition: A Survey

Wei

Liu

et al. 2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

show abstract

“…Resnet [88] introduces shortcut connections that skip one or more layers for easy optimization, accuracy gains with increasing depth and blackucing the effect of vanishing gradient problem. Multiple works [104], [118], [167], [178] employed few residual blocks as backbone to blackuce the network depth for robust recognition on small-scale ME datasets, while Lai et. al [179] introduced multi-scale shortcut connections into pre-trained deep networks to solve the gradient disappearance problem.…”

Section: A Network Blockmentioning

confidence: 99%

“…For example, inner brow raiser and outter brow raiser usually exhibits simultaneously to indicate surprise. [56] and [178] exploblack covariance correlation of local regions and multi-head self-attention to emphasis the spatial relationships, respectively. Besides spatial information, the temporal change is also important for ME.…”

Section: A Network Blockmentioning

confidence: 99%

Deep Learning for Micro-expression Recognition: A Survey

Li¹,

Wei²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Micro-expressions (MEs) are involuntary facial movements revealing people's hidden feelings in high-stake situations and have practical importance in medical treatment, national security, interrogations and many human-computer interaction systems. Early methods for MER mainly based on traditional appearance and geometry features. Recently, with the success of deep learning (DL) in various fields, neural networks have received increasing interests in MER. Different from macro-expressions, MEs are spontaneous, subtle, and rapid facial movements, leading to difficult data collection, thus have small-scale datasets. DL based MER becomes challenging due to above ME characters. To data, various DL approaches have been proposed to solve the ME issues and improve MER performance. In this survey, we provide a comprehensive review of deep microexpression recognition (MER), including datasets, deep MER pipeline, and the bench-marking of most influential methods. This survey defines a new taxonomy for the field, encompassing all aspects of MER based on DL. For each aspect, the basic approaches and advanced developments are summarized and discussed. In addition, we conclude the remaining challenges and and potential directions for the design of robust deep MER systems. To the best of our knowledge, this is the first survey of deep MER methods, and this survey can serve as a reference point for future MER research.

show abstract

“…e core idea of the optical flow-based method is to extract the information of the optical flow in the vertex frame and the start frame of the microexpression segment and then compare and analyse them. Wen et al [3] proposed a combination of traditional methods and deep learning methods for the problem of low recognition rate of crosslibrary microexpressions, in which Apex frame localization is performed in the image preprocessing part; in the feature extraction part, the TVL1 information of Apex frames is first calculated, and then the horizontal and vertical optical flow component features are fused; finally, SVM is used to classify the features. is method has a great improvement over the LPB-TOP (local binary patterns from three orthogonal planes, LPB-TOP) method.…”

Section: Introduction E Term Microexpression Was Introduced In 1996 Bymentioning

confidence: 99%

Research on a Microexpression Recognition Technology Based on Multimodal Fusion

et al. 2021

View full text Add to dashboard Cite

Microexpressions have extremely high due value in national security, public safety, medical, and other fields. However, microexpressions have characteristics that are obviously different from macroexpressions, such as short duration and weak changes, which greatly increase the difficulty of microexpression recognition work. In this paper, we propose a microexpression recognition method based on multimodal fusion through a comparative study of traditional microexpression recognition algorithms such as LBP algorithm and CNN and LSTM deep learning algorithms. The method couples the separate microexpression image information with the corresponding body temperature information to establish a multimodal fusion microexpression database. This paper firstly introduces how to build a multimodal fusion microexpression database in a laboratory environment, secondly compares the recognition accuracy of LBP, LSTM, and CNN + LSTM networks for microexpressions, and finally selects the superior CNN + LSTM network in the comparison results for model training and testing on the test set under separate microexpression database and multimodal fusion database. The experimental results show that a microexpression recognition method based on multimodal fusion designed in this paper is more accurate than unimodal recognition in multimodal recognition after feature fusion, and its recognition rate reaches 75.1%, which proves that the method is feasible and effective in improving microexpression recognition rate and has good practical value.

show abstract

Cross-database Micro Expression Recognition Based on Apex Frame Optical Flow and Multi-head Self-attention

Cited by 4 publications

References 22 publications

Deep Learning for Micro-Expression Recognition: A Survey

Deep Learning for Micro-Expression Recognition: A Survey

Deep Learning for Micro-expression Recognition: A Survey

Research on a Microexpression Recognition Technology Based on Multimodal Fusion

Contact Info

Product

Resources

About