Domain-Specific Bias Filtering for Single Labeled Domain Generalization

Abstract

Domain generalization (DG) utilizes multiple labeled source datasets to train a generalizable model for unseen target domains. However, due to expensive annotation costs, the requirements of labeling all the source data are hard to be met in realworld applications. In this paper, we investigate a Single Labeled Domain Generalization (SLDG) task with only one source domain being labeled, which is more practical and challenging than the Conventional Domain Generalization (CDG). A major obstacle in the SLDG task is the discriminability-generalization bias: discriminative information in the labeled source dataset may contain domain-specific bias, constraining the generalization of the trained model. To tackle this challenging task, we propose a novel method called Domain-Specific Bias Filtering (DSBF), which initializes a discriminative model with the labeled source data and filters out its domain-specific bias with the unlabeled source data for generalization improvement. We divide the filtering process into: (1) Feature extractor debiasing using k-means clustering-based semantic feature re-extraction; and (2) Classifier calibrating using attention-guided semantic feature projection. DSBF unifies the exploration of the labeled and the unlabeled source data to enhance the discriminability and generalization of the trained model, resulting in a highly generalizable model. We further provide theoretical analysis to verify the proposed domain-specific bias filtering process. Extensive experiments on multiple datasets show the superior performance of DSBF in tackling both the challenging SLDG task and the CDG task.

Attention-based Cross-Layer Domain Alignment for Unsupervised Domain Adaptation

“…Unsupervised domain adaptation (UDA) [3,4,5,12,13,14,15,16,17,18,19,20,21,22,23,24] aims to adapt the model trained on a labeled source domain to an unlabeled target domain when there is distinct domain divergence. A series of UDA algorithms [25,26,27,28,29,30,31] have been proposed by employing an adversarial learning strategy where the semantic features of the source and target data are aligned for reducing domain divergence.…”

Label-Efficient Domain Generalization via Collaborative Exploration and Generalization

“…Semi-Supervised Domain Generalization (SSDG) [36,47,59,65,70] aims to reduce the reliance of DG on annotation via pseudolabeling [59], consistency learning [70], or bias filtering [65]. For example, StyleMatch [70] combines consistency learning, model uncertainty learning, and style augmentation to utilize the annotation for improving model robustness.…”

“…assumption, hence may not be favorably extended to the generalization scenarios under distinct domain shifts. Semi-supervised domain generalization (SSDG) [36,47,59,65,70] tackles domain shift under the SSL setting. But some data directly assumed to be labeled in this task might not be helpful for generalization improvement but could increase the annotation costs.…”