Adversarial Graph Representation Adaptation for Cross-Domain Facial Expression Recognition

Abstract

Data inconsistency and bias are inevitable among different facial expression recognition (FER) datasets due to subjective annotating process and different collecting conditions. Recent works resort to adversarial mechanisms that learn domain-invariant features to mitigate domain shift. However, most of these works focus on holistic feature adaptation, and they ignore local features that are more transferable across different datasets. Moreover, local features carry more detailed and discriminative content for expression recognition, and thus integrating local features may enable finegrained adaptation. In this work, we propose a novel Adversarial Graph Representation Adaptation (AGRA) framework that unifies graph representation propagation with adversarial learning for cross-domain holistic-local feature co-adaptation. To achieve this, we first build a graph to correlate holistic and local regions within each domain and another graph to correlate these regions across different domains. Then, we learn the per-class statistical distribution of each domain and extract holistic-local features from the input image to initialize the corresponding graph nodes. Finally, we introduce two stacked graph convolution networks to propagate holistic-local feature within each domain to explore their interaction and across different domains for holistic-local feature co-adaptation. In this way, the AGRA framework can adaptively learn fine-grained domain-invariant features and thus facilitate cross-domain expression recognition. We conduct extensive and fair experiments on several popular benchmarks and show that the proposed AGRA framework achieves superior performance over previous state-of-the-art methods.

JDMAN: Joint Discriminative and Mutual Adaptation Networks for Cross-Domain Facial Expression Recognition

“…Even so, our JDMAN can achieve promising performance on them compared with other methods. For instance, JDMAN obtains an accuracy of 52.4% and 58.63%, which are comparable with those of AGRA [34]. It is worth noting that our method is much faster to train than AGRA.…”

“…However, the performance of ECAN is unsatisfactory to generate pseudo labels of target data, which is caused by the semantic gap between learned features and emotional labels. Xie et al proposed [34] AGRA to directly learn domain-invariant representations in an adversarial manner. In AGRA, intra-and inter-domain graphs are constructed to correlate holistic-local features within each domain and across different domains, respectively.…”

“…We compare our JDMAN with the methods specially designed for cross-domain FER, such as DETN [15], ECAN [18], and AGRA [34]. Compared with DETN [15], our method has significantly improved the result by 10.79%.…”

“…It is confused whether the improvement is caused by the different source data or methods. To make a fair comparison, we mainly compare our method with the methods which utilize RAF-DB2.0 as source data, i.e, ECAN [18] and AGRA [34]. The accuracy of ECAN on JAFFE is 61.94%.…”

“…This is because we jointly bridge domain shift and semantic gap, while ECAN only takes care of the domain discrepancy. We evaluate AGRA [34] using RAF-DB2.0 as source data by its released code. As shown in Table 1, the performance of JDMAN is 6.1% better than that of AGRA on JAFFE.…”