Theoretical Analysis of Domain Adaptation with Optimal Transport

Abstract

Domain adaptation (DA) is an important and emerging field of machine learning that tackles the problem occurring when the distributions of training (source domain) and test (target domain) data are similar but different. This kind of learning paradigm is of vital importance for future advances as it allows a learner to generalize the knowledge across different tasks. Current theoretical results show that the efficiency of DA algorithms depends on their capacity of minimizing the divergence between source and target probability distributions. In this paper, we provide a theoretical study on the advantages that concepts borrowed from optimal transportation theory [17] can bring to DA. In particular, we show that the Wasserstein metric can be used as a divergence measure between distributions to obtain generalization guarantees for three different learning settings: (i) classic DA with unsupervised target data (ii) DA combining source and target labeled data, (iii) multiple source DA. Based on the obtained results, we motivate the use of the regularized optimal transport and provide some algorithmic insights for multi-source domain adaptation. We also show when this theoretical analysis can lead to tighter inequalities than those of other existing frameworks. We believe that these results open the door to novel ideas and directions for DA.

A Review of Domain Adaptation without Target Labels

“…is a Dirac function and P i is the probability mass assigned to the i-th sample [138]. The set Γ becomes the set of all non-negative matrices of size n × m where the sum over columns corresponds to the empirical source distributionp S and the sum over rows corresponds to the empirical target distributionp T [11].…”

DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation

“…Optimal transport in domain adaptation. Optimal transport [22][23][24] has been used in domain adaptation to learn the transformation between domains [4,25,26], with associated theoretical guarantees [27]. In those works, the coupling γ is used to transport (i.e.…”

Feature Selection for Unsupervised Domain Adaptation Using Optimal Transport

“…The use of optimal transport in domain adaptation was first theoretically analyzed in [18]. In this paper, the authors proved that under some mild assumptions imposed on the form of the transport cost function, the source and target error function can be related through the following inequality where λ is the combined error of the ideal hypothesis h * that minimizes S (h) + T (h).…”