Learning two groups of discriminative features for micro-expression recognition

Wei, Jinsheng; Lu, Guanming; Yan, Jun; Zong, Yuan

doi:10.1016/j.neucom.2021.12.088

Cited by 12 publications

(4 citation statements)

References 60 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, the effectiveness of the proposed method is confirmed by comparing it with other advanced MER methods: Bi-WOOF [8], DSSN [42], Micro-attention [38], Dynamic [43], GEME [44], LFM [45], Sparse Trans [22], LFV, Trans [25], MFS Trans [46], KTGSL [47], AMAN [48], MSMMT [49], and FDCN [50]. The comparison results are depicted in Table 6.…”

Section: The Comparison Experiments With Other Approachesmentioning

confidence: 93%

Dual-Branch Cross-Attention Network for Micro-Expression Recognition with Transformer Variants

Xie,

Zhao

2024

Electronics

View full text Add to dashboard Cite

A micro-expression (ME), as a spontaneous facial expression, usually occurs instantaneously and is difficult to disguise after an emotion-evoking event. Numerous convolutional neural network (CNN)-based models have been widely explored to recognize MEs for their strong local feature representation ability on images. However, the main drawback of the current methods is their inability to fully extracting holistic contextual information from ME images. To achieve efficient ME learning representation from diverse perspectives, this paper uses Transformer variants as the main backbone and the dual-branch architecture as the main framework to extract meaningful multi-modal contextual features for ME recognition (MER). The first branch leverages an optical flow operator to facilitate the motion information extraction between ME sequences, and the corresponding optical flow maps are fed into the Swin Transformer to acquire motion–spatial representation. The second branch directly sends the apex frame in one ME clip to Mobile ViT (Vision Transformer), which can capture the local–global features of MEs. More importantly, to achieve the optimal feature stream fusion, a CAB (cross attention block) is designed to interact the feature extracted by each branch for adaptive learning fusion. The extensive experimental comparisons on three publicly available ME benchmarks show that the proposed method outperforms the existing MER methods and achieves an accuracy of 81.6% on the combined database.

show abstract

Section: The Comparison Experiments With Other Approachesmentioning

confidence: 93%

Dual-Branch Cross-Attention Network for Micro-Expression Recognition with Transformer Variants

Xie,

Zhao

2024

Electronics

View full text Add to dashboard Cite

show abstract

“…The Local Binary Pattern Six Interception Points (LBP-SIP) ( Wang et al, 2014 ) and Local Binary Pattern from Mean Orthogonal Planes (LBP-MOP) ( Wang et al, 2015 ) are used to reduce the redundancy problem. The Kernelized Two-Groups Sparse Learning (KTGSL) ( Wei et al, 2022b ) automatically learns more discriminative features from Local Binary Pattern with Single Direction Gradient (LBP-SDG) ( Wei et al, 2021 ) and Local Binary Pattern from Five Intersecting Planes (LBP-FIP) ( Wei et al, 2022a ) two sets of features to improve micro-expression recognition performance. The Discriminative Spatiotemporal Local Radon Binary Pattern Based on Revisited Integral Projection (DiSTLBP-RIP) ( Huang et al, 2019 ) fuses shape features into LBP-TOP to improve the ability to discriminate micro-expressions.…”

Section: Related Researchmentioning

confidence: 99%

Multi-scale fusion visual attention network for facial micro-expression recognition

et al. 2023

View full text Add to dashboard Cite

IntroductionMicro-expressions are facial muscle movements that hide genuine emotions. In response to the challenge of micro-expression low-intensity, recent studies have attempted to locate localized areas of facial muscle movement. However, this ignores the feature redundancy caused by the inaccurate locating of the regions of interest.MethodsThis paper proposes a novel multi-scale fusion visual attention network (MFVAN), which learns multi-scale local attention weights to mask regions of redundancy features. Specifically, this model extracts the multi-scale features of the apex frame in the micro-expression video clips by convolutional neural networks. The attention mechanism focuses on the weights of local region features in the multi-scale feature maps. Then, we mask operate redundancy regions in multi-scale features and fuse local features with high attention weights for micro-expression recognition. The self-supervision and transfer learning reduce the influence of individual identity attributes and increase the robustness of multi-scale feature maps. Finally, the multi-scale classification loss, mask loss, and removing individual identity attributes loss joint to optimize the model.ResultsThe proposed MFVAN method is evaluated on SMIC, CASME II, SAMM, and 3DB-Combined datasets that achieve state-of-the-art performance. The experimental results show that focusing on local at the multi-scale contributes to micro-expression recognition.DiscussionThis paper proposed MFVAN model is the first to combine image generation with visual attention mechanisms to solve the combination challenge problem of individual identity attribute interference and low-intensity facial muscle movements. Meanwhile, the MFVAN model reveal the impact of individual attributes on the localization of local ROIs. The experimental results show that a multi-scale fusion visual attention network contributes to micro-expression recognition.

show abstract

“…Table 2 shows the evaluated results. As a whole, the performance degrades no matter which loss is removed, thereby demonstrating the positive effects of the [19] 70.78 0.729 57.35 0.464 LFM [12] 73.98 0.717 N/A N/A LGCcon [20] 65.02 0.640 40.90 0.340 G-TCN [9] 73.98 0.725 75.00 0.699 KTGSL [21] 72.58 0.682 56.11 0.493 DeRe-GRL 80.16 0.796 72.06 0.699 three losses. In fact, the three losses are crucial to extracting the compact action features and learning the weights that can build the relationship between these action features.…”

Section: Ablation Studymentioning

confidence: 99%

Prior Information Based Decomposition and Reconstruction Learning for Micro-Expression Recognition

WEI,

CHEN,

et al. 2023

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

Micro-expression recognition (MER) draws intensive research interest as micro-expressions (MEs) can infer genuine emotions. Prior information can guide the model to learn discriminative ME features effectively. However, most works focus on researching the general models with a stronger representation ability to adaptively aggregate ME movement information in a holistic way, which may ignore the prior information and properties of MEs. To solve this issue, driven by the prior information that the category of ME can be inferred by the relationship between the actions of facial different components, this work designs a novel model that can conform to this prior information and learn ME movement features in an interpretable way. Specifically, this paper proposes a Decomposition and Reconstruction-based Graph Representation Learning (DeRe-GRL) model to efectively learn high-level ME features. DeRe-GRL includes two modules: Action Decomposition Module (ADM) and Relation Reconstruction Module (RRM), where ADM learns action features of facial key components and RRM explores the relationship between these action features. Based on facial key components, ADM divides the geometric movement features extracted by the graph model-based backbone into several subfeatures, and learns the map matrix to map these sub-features into multiple action features; then, RRM learns weights to weight all action features to build the relationship between action features. The experimental results demonstrate the effectiveness of the proposed modules, and the proposed method achieves competitive performance.

show abstract

Learning two groups of discriminative features for micro-expression recognition

Cited by 12 publications

References 60 publications

Dual-Branch Cross-Attention Network for Micro-Expression Recognition with Transformer Variants

Dual-Branch Cross-Attention Network for Micro-Expression Recognition with Transformer Variants

Multi-scale fusion visual attention network for facial micro-expression recognition

Prior Information Based Decomposition and Reconstruction Learning for Micro-Expression Recognition

Contact Info

Product

Resources

About