On Minimum Discrepancy Estimation for Deep Domain Adaptation

Abstract

In the presence of large sets of labeled data, Deep Learning (DL) has accomplished extraordinary triumphs in the avenue of computer vision, particularly in object classification and recognition tasks. However, DL cannot always perform well when the training and testing images come from different distributions or in the presence of domain shift between training and testing images. They also suffer in the absence of labeled input data. Domain adaptation (DA) methods have been proposed to make up the poor performance due to domain shift. In this paper, we present a new unsupervised deep domain adaptation method based on the alignment of second order statistics (covariances) as well as maximum mean discrepancy of the source and target data with a two stream Convolutional Neural Network (CNN). We demonstrate the ability of the proposed approach to achieve state-of-the-art performance for image classification on three benchmark domain adaptation datasets: Office-31 [27], Office-Home [37] and Office-Caltech [8].

Source-Free Domain Adaptation with Temporal Imputation for Time Series Data

“…• High-order Maximum Mean Discrepancy (HoMM) [4]: aligns the high-order moments to effectively tackle the discrepancy between the two domains. • Minimum Discrepancy Estimation for Deep Domain Adaptation (MMDA) [27]: combines the MMD and correlation alignment with entropy minimization to effectively address the domain shift issue. • Domain-Adversarial Training of Neural Networks (DANN) [9]: leverages gradient reversal layer to adversarially train a domain discriminator network against an encoder network.…”

Deep visual unsupervised domain adaptation for classification tasks: a survey

“…They jointly learned both instance‐based and centre‐based discriminative learning scheme for deep domain adaptation. Other work on this topic [21] presents an unsupervised deep domain adaptation method based on CORAL and MMD. This method by jointly utilising MMD and CORAL loss layers in the last two layers of source network and target network, aligns the second‐order statistics and higher‐order statistics, respectively.…”

Cross-Machine Fault Diagnosis with Semi-Supervised Discriminative Adversarial Domain Adaptation

“…To achieve high prediction certainty, the entropy minimization pushes the examples to nearby examples far from the decision boundary. The entropy could be minimized directly [ 36 , 37 ] or by an adversarial way [ 38 , 39 ]. In [ 36 ], Long et al imposed conditional entropy loss on target domain data, which ensures that the target classifier fits the target-specific structures well.…”