Deep Co-Training with Task Decomposition for Semi-Supervised Domain Adaptation

Yang, Lu; Wang, Yan; Gao, Mingfei; Shrivastava, Abhinav; Weinberger, Kilian Q.; Chao, Weilun; Lim, Ser-Nam

doi:10.48550/arxiv.2007.12684

Cited by 11 publications

(27 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared with the competing approaches using Resnet34 as the backbone, CDAC also achieves the best results in all cases and surpasses the current best results by 6% and 4.3% in the settings of 1-shot and 3-shot. Note that "MiCo" proposed in [42] is an unpublished work concurrent with ours, and the average performance of our CDAC using ResNet34 as the backbone is 0.4% higher than "MiCo" under the 3-shot setting.…”

Section: Comparisons With the State-of-the-artsmentioning

confidence: 81%

“…Semi-supervised domain adaptation (SSDA) is a relatively promising form of transfer learning, which intents to leverage a small number of labeled samples (e.g, one or few samples per class) in the target domain and give full play to their potential to greatly improve the performance of domain adaptation. Recently, SSDA has recently attracted wide attentions [32,29,15,21,17,42] from researchers. [32,29] first proposed to solve SSDA by align-ing the features from both domains by means of adversarial learning.…”

Section: Semi-supervised Domain Adaptationmentioning

confidence: 99%

“…To train our model with supervision from all labeled data from both source and target domains, we follow the practices of existing work on SSDA [32,29,15,21,17,42], and include the following standard cross-entropy loss in the training loss,…”

Section: Ns I=1mentioning

confidence: 99%

See 2 more Smart Citations

Cross-Domain Adaptive Clustering for Semi-Supervised Domain Adaptation

Li²,

Shi

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

In semi-supervised domain adaptation, a few labeled samples per class in the target domain guide features of the remaining target samples to aggregate around them. However, the trained model cannot produce a highly discriminative feature representation for the target domain because the training data is dominated by labeled samples from the source domain. This could lead to disconnection between the labeled and unlabeled target samples as well as misalignment between unlabeled target samples and the source domain. In this paper, we propose a novel approach called Cross-domain Adaptive Clustering to address this problem. To achieve both inter-domain and intra-domain adaptation, we first introduce an adversarial adaptive clustering loss to group features of unlabeled target data into clusters and perform cluster-wise feature alignment across the source and target domains. We further apply pseudo labeling to unlabeled samples in the target domain and retain pseudolabels with high confidence. Pseudo labeling expands the number of "labeled" samples in each class in the target domain, and thus produces a more robust and powerful cluster core for each class to facilitate adversarial learning. Extensive experiments on benchmark datasets, including DomainNet, Office-Home and Office, demonstrate that our proposed approach achieves the state-of-the-art performance in semi-supervised domain adaptation.

show abstract

Section: Comparisons With the State-of-the-artsmentioning

confidence: 81%

Section: Semi-supervised Domain Adaptationmentioning

confidence: 99%

See 1 more Smart Citation

Cross-Domain Adaptive Clustering for Semi-Supervised Domain Adaptation

Li²,

Shi

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…Isobe et al (2021) employ the KL divergence term as an objective term, however, their model is developed for the multi-target setting. Other relevant approaches in this area include combining semi-supervised learning and unsupervised domain adaptation to guide source classifiers (Yang et al 2020b), and using attention mechanism to assign target data points to source domains (Cui and Bollegala 2020).…”

Section: Related Workmentioning

confidence: 99%

Multiple-Source Domain Adaptation via Coordinated Domain Encoders and Paired Classifiers

Karisani¹

2022

Preprint

View full text Add to dashboard Cite

We present a novel multiple-source unsupervised model for text classification under domain shift. Our model exploits the update rates in document representations to dynamically integrate domain encoders. It also employs a probabilistic heuristic to infer the error rate in the target domain in order to pair source classifiers. Our heuristic exploits data transformation cost and the classifier accuracy in the target feature space. We have used real world scenarios of Domain Adaptation to evaluate the efficacy of our algorithm. We also used pretrained multi-layer transformers as the document encoder in the experiments to demonstrate whether the improvement achieved by domain adaptation models can be delivered by out-of-thebox language model pretraining. The experiments testify that our model is the top performing approach in this setting.

show abstract

“…Also aims to reduce the data distribution mismatch, compared with UDA, semi- supervised domain adaptation (SSDA) bridges domain discrepancy via introducing partially labeled target samples. Recently, a few methods have been proposed based on deep learning [46,32,22,35] for image classification. [46] decomposes SSDA into two sub-problems: UDA and SSL, and employ co-training [3] to exchange the expertise between two classifiers, which are trained on MixUp-ed [48] data between labeled and unlabeled data of each view.…”

Section: Related Workmentioning

confidence: 99%

Semi-supervised Domain Adaptation based on Dual-level Domain Mixing for Semantic Segmentation

Chen

Jia

et al. 2021

Preprint

View full text Add to dashboard Cite

Data-driven based approaches, in spite of great success in many tasks, have poor generalization when applied to unseen image domains, and require expensive cost of annotation especially for dense pixel prediction tasks such as semantic segmentation. Recently, both unsupervised domain adaptation (UDA) from large amounts of synthetic data and semi-supervised learning (SSL) with small set of labeled data have been studied to alleviate this issue. However, there is still a large gap on performance compared to their supervised counterparts. We focus on a more practical setting of semi-supervised domain adaptation (SSDA) where both a small set of labeled target data and large amounts of labeled source data are available. To address the task of SSDA, a novel framework based on dual-level domain mixing is proposed. The proposed framework consists of three stages. First, two kinds of data mixing methods are proposed to reduce domain gap in both region-level and sample-level respectively. We can obtain two complementary domain-mixed teachers based on dual-level mixed data from holistic and partial views respectively. Then, a student model is learned by distilling knowledge from these two teachers. Finally, pseudo labels of unlabeled data are generated in a self-training manner for another few rounds of teachers training. Extensive experimental results have demonstrated the effectiveness of our proposed framework on synthetic-to-real semantic segmentation benchmarks.

show abstract

Deep Co-Training with Task Decomposition for Semi-Supervised Domain Adaptation

Cited by 11 publications

References 55 publications

Cross-Domain Adaptive Clustering for Semi-Supervised Domain Adaptation

Cross-Domain Adaptive Clustering for Semi-Supervised Domain Adaptation

Multiple-Source Domain Adaptation via Coordinated Domain Encoders and Paired Classifiers

Semi-supervised Domain Adaptation based on Dual-level Domain Mixing for Semantic Segmentation

Contact Info

Product

Resources

About