Semi-Supervised Heterogeneous Domain Adaptation: Theory and Algorithms

Abstract

Semi-supervised heterogeneous domain adaptation (SsHeDA) aims to train a classifier for the target domain, in which only unlabeled and a small number of labeled data are available. This is done by leveraging knowledge acquired from a heterogeneous source domain. From algorithmic perspectives, several methods have been proposed to solve the SsHeDA problem; yet there is still no theoretical foundation to explain the nature of the SsHeDA problem or to guide new and better solutions. Motivated by compatibility condition in semi-supervised probably approximately correct (PAC) theory, we explain the SsHeDA problem by proving its generalization error -that is, why labeled heterogeneous source data and unlabeled target data help to reduce the target risk. Guided by our theory, we devise two algorithms as proof of concept. One, kernel heterogeneous domain alignment (KHDA), is a kernel-based algorithm; the other, joint mean embedding alignment (JMEA), is a neural network-based algorithm. When a dataset is small, KHDA's training time is less than JMEA's. When a dataset is large, JMEA is more accurate in the target domain. Comprehensive experiments with image/text classification tasks show KHDA to be the most accurate among all non-neural network baselines, and JMEA to be the most accurate among all baselines.

Multi-modal Domain Adaptation Method Based on Parameter Fusion and Two-Step Alignment

“…CWAN learns a feature transformer, a domain discriminator, and a classifier through the weighted adversarial approach, in order to achieve cross-domain adaptation. JMEA [38] is a semi-supervised heterogeneous domain adaptation (SsHeDA) algorithm, which can deal with massive datasets. In HCSA [39], structure preservation, distribution, and classification space alignment are implemented, jointly with feature representation by transferring both the source-domain representation and model knowledge.…”

Multiclass Classification With Fuzzy-Feature Observations: Theory and Algorithms

“…The experimental results illustrate that VAL outperforms than other three defuzzification methods. Therefore, (19) is used as the defuzzification function in all subsequent experiments.…”

Keypoint-Guided Optimal Transport

“…Zhang et al (2022) propose Masked OT model as a regularization term to preserve the local feature invariances between fine-tuned and pretrained graph neural network (GNN) for the fine-tuning of GNNs. Though the Masked OT (Zhang et al, 2022) shares similar spirits to the mask-based modeling in our method, our main contribution is the relation preservation for imposing the guidance of keypoints, different from (Zhang et al, 2022). For our mask-based modeling, it is utilized to impose the matching of keypoints with theoretical guarantee.…”

“…For our mask-based modeling, it is utilized to impose the matching of keypoints with theoretical guarantee. While the mask in (Zhang et al, 2022) aims to preserve the local information of finetuned from pretrained models. The motivation and design of our mask are different from those in (Zhang et al, 2022).…”

“…While the mask in (Zhang et al, 2022) aims to preserve the local information of finetuned from pretrained models. The motivation and design of our mask are different from those in (Zhang et al, 2022).…”