Dual-ATME: Dual-Branch Attention Network for Micro-Expression Recognition

Abstract

Micro-expression recognition (MER) is challenging due to the difficulty of capturing the instantaneous and subtle motion changes of micro-expressions (MEs). Early works based on hand-crafted features extracted from prior knowledge showed some promising results, but have recently been replaced by deep learning methods based on the attention mechanism. However, with limited ME sample sizes, features extracted by these methods lack discriminative ME representations, in yet-to-be improved MER performance. This paper proposes the Dual-branch Attention Network (Dual-ATME) for MER to address the problem of ineffective single-scale features representing MEs. Specifically, Dual-ATME consists of two components: Hand-crafted Attention Region Selection (HARS) and Automated Attention Region Selection (AARS). HARS uses prior knowledge to manually extract features from regions of interest (ROIs). Meanwhile, AARS is based on attention mechanisms and extracts hidden information from data automatically. Finally, through similarity comparison and feature fusion, the dual-scale features could be used to learn ME representations effectively. Experiments on spontaneous ME datasets (including CASME II, SAMM, SMIC) and their composite dataset, MEGC2019-CD, showed that Dual-ATME achieves better, or more competitive, performance than the state-of-the-art MER methods.

Dual-Branch Cross-Attention Network for Micro-Expression Recognition with Transformer Variants

“…The framework of the proposed MER model is illustrated in Figure 1. Specifically, this scheme adopts a dual-branch framework [28], where one branch sends the face image to a Swin Transformer after optical flow processing to extract the temporal-spatial information of ME, and the other branch sends the apex frame image to MobileViT to acquire the local-global information. More importantly, the multiple-mode features from the two branches interact through the CAB module for adaptive learning fusion.…”

MCNet: meta-clustering learning network for micro-expression recognition

“…In addition to handling discriminative representations of micro-expression sequences from a dynamic feature perspective, deep learning methods based on attention mechanisms have also demonstrated success. [31][32][33][34] For instance, Zhou et al 32 proposed the dual-branch attention network (dual-ATME), comprising hand-crafted attention region selection (HARS) and automated attention region selection (AARS). The HARS manually extracted features from the ROI using prior knowledge, while AARS automatically extracted hidden information from the sequence based on attention mechanisms.…”

“…In addition to handling discriminative representations of micro-expression sequences from a dynamic feature perspective, deep learning methods based on attention mechanisms have also demonstrated success 31 – 34 For instance, Zhou et al 32 . proposed the dual-branch attention network (dual-ATME), comprising hand-crafted attention region selection (HARS) and automated attention region selection (AARS).…”