Generalizing to Unseen Domains: A Survey on Domain Generalization

Wang, Jindong; Lan, Cuiling; Liu, Chang; Ouyang, Yidong; Qin, Tao

doi:10.24963/ijcai.2021/628

Cited by 195 publications

(50 citation statements)

References 123 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even though metalearning is a general learning framework, it is recently applied for Domain Generalization tasks [10] [25] [26]. For more details, we refer the reader to the recent surveys on Domain Generalization in [27] and [28].…”

Section: A Domain Generalizationmentioning

confidence: 99%

Conditional entropy minimization principle for learning domain invariant representation features

Nguyen¹,

Lyu²,

Ishwar³

et al. 2022

Preprint

View full text Add to dashboard Cite

Invariance principle-based methods, for example, Invariant Risk Minimization (IRM), have recently emerged as promising approaches for Domain Generalization (DG). Despite the promising theory, invariance principle-based approaches fail in common classification tasks due to the mixture of the true invariant features and the spurious invariant features. In this paper, we propose a framework based on the conditional entropy minimization principle to filter out the spurious invariant features leading to a new algorithm with a better generalization capability. We theoretically prove that under some particular assumptions, the representation function can precisely recover the true invariant features. In addition, we also show that the proposed approach is closely related to the well-known Information Bottleneck framework. Both the theoretical and numerical results are provided to justify our approach.

show abstract

Section: A Domain Generalizationmentioning

confidence: 99%

Conditional entropy minimization principle for learning domain invariant representation features

Nguyen¹,

Lyu²,

Ishwar³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Theorem 1 (Average risk estimation error bound for binary classification [10], [12]). Assume that the loss function is L -Lipschitz in its first argument and is bounded by B .…”

Section: Error Bounds For Mdgmentioning

confidence: 99%

“…The simplest solution to this problem is the vanilla empirical risk minimization (ERM) [11], which minimizes an average loss computed on data pooled together from all available source domains. The inability of this approach to exploit statistical discrepancies between domains has motivated the design of multi-domain learning techniques [12]. However, recently Gulrajani et al [13] reported that a powerful feature extractor coupled with effective model selection can make ERMs highly competitive on standard benchmarks.…”

Section: Introductionmentioning

confidence: 99%

Improving Multi-Domain Generalization through Domain Re-labeling

Thopalli¹,

Katoch²,

Spanias³

et al. 2021

Preprint

View full text Add to dashboard Cite

Domain generalization (DG) methods aim to develop models that generalize to settings where the test distribution is different from the training data. In this paper, we focus on the challenging problem of multi-source zero shot DG, where labeled training data from multiple source domains is available but with no access to data from the target domain. Though this problem has become an important topic of research, surprisingly, the simple solution of pooling all source data together and training a single classifier is highly competitive on standard benchmarks. More importantly, even sophisticated approaches that explicitly optimize for invariance across different domains do not necessarily provide non-trivial gains over ERM. In this paper, for the first time, we study the important link between pre-specified domain labels and the generalization performance. Using a motivating case-study and a new variant of a distributional robust optimization algorithm, GroupDRO++, we first demonstrate how inferring custom domain groups can lead to consistent improvements over the original domain labels that come with the dataset. Subsequently, we introduce a general approach for multi-domain generalization, MulDEns, that uses an ERM-based deep ensembling backbone and performs implicit domain re-labeling through a meta-optimization algorithm. Using empirical studies on multiple standard benchmarks, we show that MulDEns does not require tailoring the augmentation strategy or the training process specific to a dataset, consistently outperforms ERM by significant margins, and produces stateof-the-art generalization performance, even when compared to existing methods that exploit the domain labels.

show abstract

“…Researchers have proposed many domain adaptation methods to address the above-mentioned problem [ 19 , 20 , 21 , 22 , 23 ]. CycleGAN [ 13 ] is a representative type of pixel-level domain adaptation method, which transforms the source domain data in the original pixel space into a style in the target domain, capturing pixel-level and low-level domain shifts.…”

Section: Introductionmentioning

confidence: 99%

Wildfire Smoke Classification Based on Synthetic Images and Pixel- and Feature-Level Domain Adaptation

Mao

Zheng

Yin

et al. 2021

Sensors

View full text Add to dashboard Cite

Training a deep learning-based classification model for early wildfire smoke images requires a large amount of rich data. However, due to the episodic nature of fire events, it is difficult to obtain wildfire smoke image data, and most of the samples in public datasets suffer from a lack of diversity. To address these issues, a method using synthetic images to train a deep learning classification model for real wildfire smoke was proposed in this paper. Firstly, we constructed a synthetic dataset by simulating a large amount of morphologically rich smoke in 3D modeling software and rendering the virtual smoke against many virtual wildland background images with rich environmental diversity. Secondly, to better use the synthetic data to train a wildfire smoke image classifier, we applied both pixel-level domain adaptation and feature-level domain adaptation. The CycleGAN-based pixel-level domain adaptation method for image translation was employed. On top of this, the feature-level domain adaptation method incorporated ADDA with DeepCORAL was adopted to further reduce the domain shift between the synthetic and real data. The proposed method was evaluated and compared on a test set of real wildfire smoke and achieved an accuracy of 97.39%. The method is applicable to wildfire smoke classification tasks based on RGB single-frame images and would also contribute to training image classification models without sufficient data.

show abstract

Generalizing to Unseen Domains: A Survey on Domain Generalization

Cited by 195 publications

References 123 publications

Conditional entropy minimization principle for learning domain invariant representation features

Conditional entropy minimization principle for learning domain invariant representation features

Improving Multi-Domain Generalization through Domain Re-labeling

Wildfire Smoke Classification Based on Synthetic Images and Pixel- and Feature-Level Domain Adaptation

Contact Info

Product

Resources

About