Unsupervised Domain Adaptation with Similarity Learning

Abstract

The objective of unsupervised domain adaptation is to leverage features from a labeled source domain and learn a classifier for an unlabeled target domain, with a similar but different data distribution. Most deep learning approaches to domain adaptation consist of two steps: (i) learn features that preserve a low risk on labeled samples (source domain) and (ii) make the features from both domains to be as indistinguishable as possible, so that a classifier trained on the source can also be applied on the target domain. In general, the classifiers in step (i) consist of fully-connected layers applied directly on the indistinguishable features learned in (ii). In this paper, we propose a different way to do the classification, using similarity learning. The proposed method learns a pairwise similarity function in which classification can be performed by computing similarity between prototype representations of each category. The domain-invariant features and the categorical prototype representations are learned jointly and in an end-to-end fashion. At inference time, images from the target domain are compared to the prototypes and the label associated with the one that best matches the image is outputed. The approach is simple, scalable and effective. We show that our model achieves state-of-the-art performance in different unsupervised domain adaptation scenarios.

Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation

“…Under this motivation, multiple methods have been used to align the distributions of two domains, such as maximum mean discrepancy (MMD) [6,7,18], CORrelation ALignment (CORAL) [8,21], attention [22], and optimal transport [23]. Besides, adversarial learning is also used to learn domain-invariant features [9,10,20,24]. On par with these methods aligning distributions in the feature space, some methods align distributions in raw pixel space by translating source data to the target domain with Image to Image translation techniques [25][26][27][28][29][30].…”

“…Furthermore, the second term is also expected to be small by optimizing the domain-invariant features between S and T . The third term is treated as a negligibly small term and is usually disregarded by previous methods [7,9,20]. However, a large C may hurt the performance on the target domain [43].…”

“…The Digits datasets include USPS [47] (U domain) and MNIST [48] (M domain). For the tasks U→M and M→U, we conduct the experiments on two experimental settings: 1) using all the training data in MNIST and USPS during training [20,26]; 2) sampling 2,000 training samples from MNIST and 1,800 training samples from USPS for training [10].…”

“…Baseline Methods. We compared our proposed SDA-TCL with state-of-the-art methods: (I) Domainlevel distribution alignment methods: Gradient Reversal (RevGrad) [9], Similarity Learning (SimNet) [20]; (II) Class-level distribution alignment methods: Transferable Prototypical Networks (TPN) [12], Domain-Invariant Adversarial Learning (DIAL) [14], Similarity Constrained Alignment (SCA) [15]; (III) Aligning distributions on pixel-level methods: Coupled Generative Adversarial Network (CoGAN) [25], Cycle-Consistent Adversarial Domain Adaptation (CyCADA) [29], Generate To Adapt (GTA) [30]; (IV) Utilizing pseudo-labels implicitly methods: Maximum Classifier Discrepancy (MCD) [35]; (V)Learning target classifier methods: Incremental Collaborative and Adversarial Network (iCAN) [17]; (V)State-of-the-art Methods: Joint Adaptation Network (JAN) [18], Deep adversarial Attention Alignment (DAAA) [22], Self-Ensembling (S-En) [38], Conditional Domain Adversarial Networks (CDAN+E) [24]. With the same protocol, we cite the results from the papers respectively following the previous methods [11,16].…”

AITL: Adversarial Inductive Transfer Learning with input and output space adaptation for pharmacogenomics

“…In the context of drug response prediction, Mourragui et al (Mourragui et al, 2019) proposed PRECISE, a subspace-centric method, based on principal component analysis to minimize the discrepancy in the input space between cell lines and patients. Recently, adversarial domain adaptation has shown great performance in addressing the discrepancy in the input space for different applications, and its performance is comparable to the metric-based and subspace-centric methods in computer vision (Hosseini-Asl et al, 2018;Pinheiro, 2018;Zou et al, 2018;Tsai et al, 2018;Long et al, 2018;Chen et al, 2017;Tzeng et al, 2017;Ganin and Lempitsky, 2014). However, adversarial adaptation that addresses the discrepancies in both the input and output spaces has not yet been explored neither for pharmacogenomics nor for other applications.…”