Joint Patch Weighting and Moment Matching for Unsupervised Domain Adaptation in Micro-Expression Recognition

Abstract

Unsupervised domain adaptation (DA) is a challenging machine learning problem since the labeled training (source) and unlabeled testing (target) sets belong to different domains and then have different feature distributions, which has recently attracted wide attention in micro-expression recognition (MER). Although some well-performing unsupervised DA methods have been proposed, these methods cannot well solve the problem of unsupervised DA in MER, a. k. a., cross-domain MER. To deal with such a challenging problem, in this letter we propose a novel unsupervised DA method called Joint Patch weighting and Moment Matching (JPMM). JPMM bridges the source and target micro-expression feature sets by minimizing their probability distribution divergence with a multi-order moment matching operation. Meanwhile, it takes advantage of the contributive facial patches by the weight learning such that a domaininvariant feature representation involving micro-expression distinguishable information can be learned. Finally, we carry out extensive experiments to evaluate the proposed JPMM method is superior to recent state-of-the-art unsupervised DA methods in dealing with cross-domain MER.

Micro-Expression Recognition Based on Spatio-Temporal Feature Extraction of Key Regions

“…The extraction sequence numbers of 72 facial key points (see Fig. 2b) are: 19,22,23,26,39,37,44,46,28,30,49,51,53,55,59 and 57. The serial numbers of the IPs generated (see Fig.…”

“…Different AUs represent different local facial actions. For example, AU1 represents the inner browser raiser, while AU5 represents the upper lip raiser [17][18][19]. ME generation is usually the result of the joint action of one or more AUs.…”

“…The calculation of UAR is shown in Eq. (19), where TP i is the number of correct predictions in the i-th category, and N i is the number of samples in the i-th sample. Acc is shown in Eq.…”