JDMAN: Joint Discriminative and Mutual Adaptation Networks for Cross-Domain Facial Expression Recognition

Abstract

Cross-domain Facial Expression Recognition (FER) is challenging due to the difficulty of concurrently handling the domain shift and semantic gap during domain adaptation. Existing methods mainly focus on reducing the domain discrepancy for transferable features but fail to decrease the semantic one, which may result in negative transfer. To this end, we propose Joint Discriminative and Mutual Adaptation Networks (JDMAN), which collaboratively bridge the domain shift and semantic gap by domain-and category-level coadaptation based on mutual information and discriminative metric learning techniques. Specifically, we design a mutual information minimization module for domain-level adaptation, which narrows the domain shift by simultaneously distilling the domain-invariant components and eliminating the untransferable ones lying in different domains. Moreover, we propose a semantic metric learning module for category-level adaptation, which can close the semantic discrepancy during discriminative intra-domain representation learning and transferable inter-domain knowledge discovery. These two modules are jointly leveraged in our JDMAN to safely transfer the source knowledge to target data in an end-to-end manner. Extensive experimental results on six databases show that our method achieves state-of-the-art performance. The code of our JDMAN is available at https://github.com/YingjianLi/JDMAN. CCS CONCEPTS• Computing methodologies → Computer vision.

Transferable driver facial expression recognition based on joint discriminative correlation alignment network with enhanced feature attention

“…Consequently, when the feature distribution of the target domain is different from that of the source domain, the performance of facial expression will suffer from degeneration. Therefore, several transfer learning or domain adaptation methods [24][25][26][27] have been proposed to exploit the knowledge learned from the source domain to promote the learning task in target domain. The most common transfer learning approach is to train a model in one domain and then fine-tune it in a related domain.…”