Learning to Optimize Domain Specific Normalization for Domain Generalization

Seo, Seonguk; Suh, Young-Ger; Kim, Dong-Wan; Han, Jong-Woo; Han, Bohyung

doi:10.48550/arxiv.1907.04275

Cited by 14 publications

(28 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Domain generalization (DG) considers the generalization ability to novel domains of deep models trained on source domains where the heterogeneity caused by domain shifts is significant. A common approach is extracting domain-invariant features over multiple source domains [11,16,28,30,33,35,40,45,49,63] or aggregating domain-specific modules [37,38] to conduct domain-invariant or domain-specific. Many works propose to enlarge the available data space with augmentation of source domains [4,12,46,51,64,65].…”

Section: Related Workmentioning

confidence: 99%

Towards Unsupervised Domain Generalization

Zhang¹,

Zhou²,

Xu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Domain generalization (DG) aims to help models trained on a set of source domains generalize better on unseen target domains. The performances of current DG methods largely rely on sufficient labeled data, which however are usually costly or unavailable. While unlabeled data are far more accessible, we seek to explore how unsupervised learning can help deep models generalizes across domains. Specifically, we study a novel generalization problem called unsupervised domain generalization, which aims to learn generalizable models with unlabeled data. Furthermore, we propose a Domain-Irrelevant Unsupervised Learning (DIUL) method to cope with the significant and misleading heterogeneity within unlabeled data and severe distribution shifts between source and target data. Surprisingly we observe that DIUL can not only counterbalance the scarcity of labeled data, but also further strengthen the generalization ability of models when the labeled data are sufficient. As a pretraining approach, DIUL shows superior to ImageNet pretraining protocol even when the available data are unlabeled and of a greatly smaller amount compared to ImageNet. Extensive experiments clearly demonstrate the effectiveness of our method compared with state-of-the-art unsupervised learning counterparts.Recently, the research community has spent significant effort to develop algorithms for learning visual representations without category labels [20,48,61] and shows the advance of contrastive *Corresponing author, also with Beijing Key Lab of Networked Multimedia. Preprint. Under review.

show abstract

Section: Related Workmentioning

confidence: 99%

Towards Unsupervised Domain Generalization

Zhang¹,

Zhou²,

Xu³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Domain generalization (DG) considers the generalization capacities to unseen domains of deep models trained with multiple source domains. A common approach is to extract domain-invariant features over multiple source domains [18,29,37,39,47,11,25,50,56,47] or to aggregate domain-specific modules [43,44]. Several works propose to enlarge the available data space with augmentation of source domains [7,57,67,52,73,72].…”

Section: Related Workmentioning

confidence: 99%

Deep Stable Learning for Out-Of-Distribution Generalization

Zhang¹,

Cui²,

Xu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Approaches based on deep neural networks have achieved striking performance when testing data and training data share similar distribution, but can significantly fail otherwise. Therefore, eliminating the impact of distribution shifts between training and testing data is crucial for building performance-promising deep models. Conventional methods assume either the known heterogeneity of training data (e.g. domain labels) or the approximately equal capacities of different domains. In this paper, we consider a more challenging case where neither of the above assumptions holds. We propose to address this problem by removing the dependencies between features via learning weights for training samples, which helps deep models get rid of spurious correlations and, in turn, concentrate more on the true connection between discriminative features and labels. Extensive experiments clearly demonstrate the effectiveness of our method on multiple distribution generalization benchmarks compared with state-of-the-art counterparts. Through extensive experiments on distribution generalization benchmarks including PACS, VLCS, MNIST-M, and NICO, we show the effectiveness of our method compared with state-of-the-art counterparts.

show abstract

“…[35] propose a low-rank decomposition on the final classification layer to identifiably learn common and specific features across domains. [37] use domain-specific normalizations to learn representations that are domain-agnostic and semantically discriminative. All of the above methods require domain labels, which might not always be viable.…”

Section: Prior Workmentioning

confidence: 99%

Domain Generalization via Inference-time Label-Preserving Target Projections

Pandey,

Raman,

Varambally

et al. 2021

Preprint

View full text Add to dashboard Cite

Generalization of machine learning models trained on a set of source domains on unseen target domains with different statistics, is a challenging problem. While many approaches have been proposed to solve this problem, they only utilize source data during training but do not take advantage of the fact that a single target example is available at the time of inference. Motivated by this, we propose a method that effectively uses the target sample during inference beyond mere classification. Our method has three components -(i) A label-preserving feature or metric transformation on source data such that the source samples are clustered in accordance with their class irrespective of their domain (ii) A generative model trained on the these features (iii) A label-preserving projection of the target point on the source-feature manifold during inference via solving an optimization problem on the input space of the generative model using the learned metric. Finally, the projected target is used in the classifier. Since the projected target feature comes from the source manifold and has the same label as the real target by design, the classifier is expected to perform better on it than the true target. We demonstrate that our method outperforms the state-of-the-art Domain Generalization methods on multiple datasets and tasks.

show abstract

Learning to Optimize Domain Specific Normalization for Domain Generalization

Cited by 14 publications

References 25 publications

Towards Unsupervised Domain Generalization

Towards Unsupervised Domain Generalization

Deep Stable Learning for Out-Of-Distribution Generalization

Domain Generalization via Inference-time Label-Preserving Target Projections

Contact Info

Product

Resources

About