Cross Domain Mean Approximation for Unsupervised Domain Adaptation

Abstract

Unsupervised Domain Adaptation (UDA) aims to leverage the knowledge from the labeled source domain to help the task of target domain with the unlabeled data. It is a key step for UDA to minimize the cross-domain distribution divergence. In this paper, we firstly propose a novel discrepancy metric, referred to as Cross Domain Mean Approximation (CDMA) discrepancy, to evaluate the distribution differences between source and target domains, which calculate the sum of the squares of the distances from the source and target domains to the mean of the other domain. Secondly, Joint Distribution Adaptation based on Cross Domain Mean Approximation (JDA-CDMA) is developed on the basis of CDMA to extract shared feature and simultaneously reduce the marginal and conditional distribution discrepancy between domains during the label refinement process. Thirdly, we construct a classifier utilizing CDMA metric and neighbor information. Finally, the proposed feature extraction approach and classifier are combined to realize transfer learning. Results from extensive experiments on five visual benchmarks including object, face, and digit images, show the proposed methods outperform the state-of-the-art unsupervised domain adaptation. I. INTRODUCTION I N machine vision, many machine learning methods, such as Linear Regression [1], Logistic Regression (LR) [2], k-Nearest Neighbor (k-NN) [3], Bayesian [4], Decision Tree [5], and Support Vector Machine (SVM)[6], are applied to image classification tasks. However, when the image feature representation is too redundant or poor in quality, their accuracy will be lowered. Therefore, it is of great importance to extract high-quality image feature. Feature extraction, as an important manner of mining image latent knowledge, is not only conducive to the in-depth understanding of image content, but also crucial to improve the accuracy of image classification and recognition [7]. Consequently it has attracted much attention from researchers. Principal Component Analysis (PCA) [8], Independent Component Analysis (ICA) [9], Linear Discriminant Analysis (LDA) [10], Maximum Margin Criterion (MMC) [11] and other algorithms are often used for feature extraction. In order to discover the nonlinear structure hidden in the high dimensional data and mine the local geometric structure information of data, Laplacian Eigen-maps (LE) [12], Locality Linear Embedding

TSTELM: Two-Stage Transfer Extreme Learning Machine for Unsupervised Domain Adaptation

“…Here, inspired by [ 35 ], we introduce cross-domain mean approximation to replace the prediction loss. When there are no labeled samples in the target domains (when c =0), we force target data H T close to source data mean point H S _ av , which promotes domain adaptation seen from [ 35 ]. If the target sample obtains pseudo labels, it is drawn to source data mean point with the same category H S _ av ( c ) .…”

“…Hence, we can learn the other DAELM. It is worth emphasizing that we present target cross-domain mean approximation referred to [ 35 ] to adapt the distribution of the target domain for consistency with the source domains. At prediction stage, the above two DAELMs jointly determined the category of test samples.…”

Transfer Extreme Learning Machine with Output Weight Alignment

“…In transfer learning, domain adaptation accelerates the cross-domain transfer of knowledge by minimizing the discrepancy between domains. According to “how to correct interdomain distribution mismatch,” domain adaptation can be roughly divided into three categories: sample weighting, subspace and manifold alignment, and statistical distribution alignment [ 33 ].…”

“…Domain adaptation [ 31 – 33 ], as an important branch of transfer learning, solves the above problems with the help of the knowledge from the source domain which is different from but related to the target domain and resolves the inconsistency of sample distribution between the source and target domains. Zhang and Zhang [ 34 ] extended ELM to handle domain adaptation problems with very few labeled guide samples in target domain and overcome the generalization disadvantages of ELM in multidomain application.…”

Unsupervised Domain Adaptation Based on Pseudo-Label Confidence

“…It should be pointed out that the methods mentioned above [8], [10], [23], [24] ignore a fact that samples after projection may not be discriminative enough for the final classification. And in UDA, the labels of the target domain are not available [25].…”