SENTRY: Selective Entropy Optimization via Committee Consistency for Unsupervised Domain Adaptation

Prabhu, Viraj; Khare, Shivam; Kartik, Deeksha; Hoffman, Judy

doi:10.1109/iccv48922.2021.00844

Cited by 70 publications

(29 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Acc DANN(ICML'15) [13] 57.4 MDD (ICML'19) [90] 74.6 MDD + FixMatch [39] 77.8 CDAN (NeurIPS'18) [40] 70.0 CDAN + VAT + Entropy [39] 76.5 GVB (CVPR'20) [9] 75.3 SENTRY (ICCV'21) [53] 76.7 CST (NeurIPS'21) [39] 79.9 CST + SAM (NeurIPS'21) [39] 80 the proportion of high-confidence samples in 60%-80%, where τ ∈ [0.93, 0.98] is used. For LRCo, there are three hyperparameters: temperature T co in Eq.…”

Section: Methodsmentioning

confidence: 99%

Low-confidence Samples Matter for Domain Adaptation

Zhang¹,

Li²,

Wang³

2022

Preprint

View full text Add to dashboard Cite

Domain adaptation (DA) aims to transfer knowledge from a label-rich source domain to a related but labelscarce target domain. The conventional DA strategy is to align the feature distributions of the two domains. Recently, increasing researches have focused on self-training or other semi-supervised algorithms to explore the data structure of the target domain. However, the bulk of them depend largely on confident samples in order to build reliable pseudo labels, prototypes or cluster centers. Representing the target data structure in such a way would overlook the huge low-confidence samples, resulting in suboptimal transferability that is biased towards the samples similar to the source domain. To overcome this issue, we propose a novel contrastive learning method by processing low-confidence samples, which encourages the model to make use of the target data structure through the instance discrimination process. To be specific, we create positive and negative pairs only using low-confidence samples, and then re-represent the original features with the classifier weights rather than directly utilizing them, which can better encode the task-specific semantic information. Furthermore, we combine cross-domain mixup to augment the proposed contrastive loss. Consequently, the domain gap can be well bridged through contrastive learning of intermediate representations across domains. We evaluate the proposed method in both unsupervised and semi-supervised DA settings, and extensive experimental results on benchmarks reveal that our method is effective and achieves state-ofthe-art performance. The code can be found in https: //github.com/zhyx12/MixLRCo.

show abstract

Section: Methodsmentioning

confidence: 99%

Low-confidence Samples Matter for Domain Adaptation

Zhang¹,

Li²,

Wang³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We further experiment with forming an ensemble of different augmented views of the dataset. This is inspired by SENTRY [40], which uses a committee of augmented views to guide training.…”

Section: Related Workmentioning

confidence: 99%

Pick up the PACE: Fast and Simple Domain Adaptation via Ensemble Pseudo-Labeling

Liao¹,

Tsiligkaridis²,

Kulis³

2022

Preprint

View full text Add to dashboard Cite

Domain Adaptation (DA) has received widespread attention from deep learning researchers in recent years because of its potential to improve test accuracy with out-of-distribution labeled data. Most state-of-the-art DA algorithms require an extensive amount of hyperparameter tuning and are computationally intensive due to the large batch sizes required. In this work, we propose a fast and simple DA method consisting of three stages: (1) domain alignment by covariance matching, (2) pseudo-labeling, and (3) ensembling. We call this method PACE, for Pseudo-labels, Alignment of Covariances, and Ensembles. PACE is trained on top of fixed features extracted from an ensemble of modern pretrained backbones. PACE exceeds previous state-of-the-art by 5 -10 % on most benchmark adaptation tasks without training a neural network. PACE reduces training time and hyperparameter tuning time by 82% and 97%, respectively, when compared to state-of-the-art DA methods. Code is released here: https://github.com/Chris210634/PACE-Domain-AdaptationPreprint. Under review.

show abstract

“…Domain Adaptation (DA) presents under many different settings. In the vanilla Unsupervised Domain Adaptation (UDA) [45,29,3,35], only labeled data from a source domain and unlabeled data from a target domain are available. Since no labeled target data is available, UDA can be a challenging task when the domain gap is large.…”

Section: Domain Adaptationmentioning

confidence: 99%

Prior Knowledge Guided Unsupervised Domain Adaptation

Sun¹,

Liu²,

Ling³

2022

Preprint

View full text Add to dashboard Cite

The waive of labels in the target domain makes Unsupervised Domain Adaptation (UDA) an attractive technique in many real-world applications, though it also brings great challenges as model adaptation becomes harder without labeled target data. In this paper, we address this issue by seeking compensation from target domain prior knowledge, which is often (partially) available in practice, e.g., from human expertise. This leads to a novel yet practical setting where in addition to the training data, some prior knowledge about the target class distribution are available. We term the setting as Knowledge-guided Unsupervised Domain Adaptation (KUDA). In particular, we consider two specific types of prior knowledge about the class distribution in the target domain: Unary Bound that describes the lower and upper bounds of individual class probabilities, and Binary Relationship that describes the relations between two class probabilities. We propose a general rectification module that uses such prior knowledge to refine model generated pseudo labels. The module is formulated as a Zero-One Programming problem derived from the prior knowledge and a smooth regularizer. It can be easily plugged into self-training based UDA methods, and we combine it with two state-of-the-art methods, SHOT and DINE. Empirical results on four benchmarks confirm that the rectification module clearly improves the quality of pseudo labels, which in turn benefits the self-training stage. With the guidance from prior knowledge, the performances of both methods are substantially boosted. We expect our work to inspire further investigations in integrating prior knowledge in UDA.Code is available at https://github.com/tsun/KUDA.

show abstract

SENTRY: Selective Entropy Optimization via Committee Consistency for Unsupervised Domain Adaptation

Cited by 70 publications

References 20 publications

Low-confidence Samples Matter for Domain Adaptation

Low-confidence Samples Matter for Domain Adaptation

Pick up the PACE: Fast and Simple Domain Adaptation via Ensemble Pseudo-Labeling

Prior Knowledge Guided Unsupervised Domain Adaptation

Contact Info

Product

Resources

About