Facial Expression Recognition With Two-Branch Disentangled Generative Adversarial Network

Xie, Siyue; Hu, Haifeng; Chen, Yizhen

doi:10.1109/tcsvt.2020.3024201

Cited by 51 publications

(13 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…IDFERM [5] 6 expressions and contempt 98.35 DeRL [2] 6 expressions and contempt 97.3 IA-gen [22] 6 expressions and contempt 96.57 ADFL [4] 6 expressions and contempt 98.17 TDGAN [6] 6 expressions and neutral 97.53 TER-GAN [7] 6 expressions and contempt 98.47 Huang et.al. [23] 6 expressions and contempt 98.65 IF-GAN [8] 6 expressions 97.52 Cross-VAE [35] 6 expressions and contempt 94.96 FN2EN [36] 6 expressions 98.…”

Section: Methodsmentioning

confidence: 99%

“…TDGAN [6] proposes a expression transferring framework which can generate an expression image to a given face (identity). This is achieved with two branches of encoders, one decoder and two discriminators.…”

Section: Identity-invariant Fermentioning

confidence: 99%

“…Expression transferring based methods disentangle identity and expression from a new perspective. TDGAN [6] and TER-…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Disentangling Identity and Pose for Facial Expression Recognition

Jiang

Deng

2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Facial expression recognition (FER) is a challenging problem because the expression component is always entangled with other irrelevant factors, such as identity and head pose. In this work, we propose an identity and pose disentangled facial expression recognition (IPD-FER) model to learn more discriminative feature representation. We regard the holistic facial representation as the combination of identity, pose and expression. These three components are encoded with different encoders. For identity encoder, a well pre-trained face recognition model is utilized and fixed during training, which alleviates the restriction on specific expression training data in previous works and makes the disentanglement practicable on in-the-wild datasets. At the same time, the pose and expression encoder are optimized with corresponding labels. Combining identity and pose feature, a neutral face of input individual should be generated by the decoder. When expression feature is added, the input image should be reconstructed. By comparing the difference between synthesized neutral and expressional images of the same individual, the expression component is further disentangled from identity and pose. Experimental results verify the effectiveness of our method on both lab-controlled and in-the-wild databases and we achieve state-of-the-art recognition performance.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Identity-invariant Fermentioning

confidence: 99%

See 1 more Smart Citation

Disentangling Identity and Pose for Facial Expression Recognition

Jiang

Deng

2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

show abstract

“…Method RAF-DB SFEW DLP-CNN [25] 84.13 51.05 IACNN [30] -50.98 SPDNet [1] 87.00 58.14 IPA2LT [44] 86.77 58.29 IPFR [36] -57.10 † TDGAN [41] 81.91±1.18 -RAN [38] 86.90 56.40 ‡ SCN [37] 87.03 -DDL [32] 87 [38]. and pose) are taken into account in D 3 Net.…”

Section: Comparison With State-of-the-art Fer Methodsmentioning

confidence: 99%

“…In other words, the expression information is seriously entangled with the disturbance information for the features obtained by the baseline method. In contrast, compared [30] 95.37 71.55 -PHRNN-MSCNN [47] 98.50 81.18 86.25 FN2EN [9] 98.60 ⋆ -87.71 DLP-CNN [24] 95.78 ⋆ 78.46 -IPA2LT [44] 92.45 ⋆ 65.61 61.49 DeRL [42] 97.30 73.23 88.00 ADFL [2] 98.17 77.51 87.50 TDGAN [41] 97.53±2.03 ⋆ --DDL [32] 99 with the baseline method, D 3 Net shows better inter-class separability and intra-class compactness (see Figure 3(c)). Moreover, the features corresponding to some similar expressions (i.e., sad, neutral, and angry) are also more distinguishable.…”

Section: Ablation Studiesmentioning

confidence: 99%

D³Net: Dual-Branch Disturbance Disentangling Network for Facial Expression Recognition

Yan

Xue

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

One of the main challenges in facial expression recognition (FER) is to address the disturbance caused by various disturbing factors, including common ones (such as identity, pose, and illumination) and potential ones (such as hairstyle, accessory, and occlusion). Recently, a number of FER methods have been developed to explicitly or implicitly alleviate the disturbance involved in facial images. However, these methods either consider only a few common disturbing factors or neglect the prior information of these disturbing factors, thus resulting in inferior recognition performance. In this paper, we propose a novel Dual-branch Disturbance Disentangling Network (D 3 Net), mainly consisting of an expression branch and a disturbance branch, to perform effective FER. In the disturbance branch, a label-aware sub-branch (LAS) and a label-free sub-branch (LFS) are elaborately designed to cope with different types of disturbing factors. On the one hand, LAS explicitly captures the disturbance due to some common disturbing factors by transfer learning on a pretrained model. On the other hand, LFS implicitly encodes the information of potential disturbing factors in an unsupervised manner. In particular, we introduce an Indian buffet process (IBP) prior to model the distribution of potential disturbing factors in LFS. Moreover, we leverage adversarial training to increase the differences between disturbance features and expression features, thereby enhancing the disentanglement of disturbing factors. By disentangling the disturbance from facial images, we are able to extract discriminative expression features. Extensive experiments demonstrate that our proposed method performs favorably

show abstract

Expression Recognition Using a Flow-Based Latent-Space Representation

Aathreya

Canavan

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Facial Expression Recognition With Two-Branch Disentangled Generative Adversarial Network

Cited by 51 publications

References 38 publications

Disentangling Identity and Pose for Facial Expression Recognition

Disentangling Identity and Pose for Facial Expression Recognition

D³Net: Dual-Branch Disturbance Disentangling Network for Facial Expression Recognition

Expression Recognition Using a Flow-Based Latent-Space Representation

Contact Info

Product

Resources

About