RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening

Choi, Sungha; Jung, Sung-Eun; Yun, Huiwon; Kim, Joanne T.; Kim, Seungryong; Choo, Jaegul

doi:10.1109/cvpr46437.2021.01141

Cited by 179 publications

(139 citation statements)

References 56 publications

Supporting

Mentioning

139

Contrasting

Order By: Relevance

“…Instead of enforcing the entire features to be invariant, disentangled feature learning approaches (Chattopadhyay et al (2020); Piratla et al (2020)) decouple the features into domain-specific and domain-invariant parts and learn their representations simultaneously. In addition, normalizationbased methods (Pan et al (2018); Choi et al (2021)) can also be used to remove the style information to obtain invariant representations.…”

Section: Invariant Representation Learningmentioning

confidence: 99%

“…This assumption, however, does not hold in many real-world applications. For instance, when employing segmentation models trained on sunny days for rainy and foggy environments (Choi et al, 2021), or recognizing art paintings with models that trained on photographs (Li et al, 2017), inevitable performance drop can often be observed in such out-of-distribution deployment scenarios. Therefore, the problem of domain generalization, aiming to improve the robustness of the network on various unseen testing domains, becomes quite important.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Uncertainty Modeling for Out-of-Distribution Generalization

Li¹,

Dai²,

Ge³

et al. 2022

Preprint

View full text Add to dashboard Cite

Though remarkable progress has been achieved in various vision tasks, deep neural networks still suffer obvious performance degradation when tested in out-ofdistribution scenarios. We argue that the feature statistics (mean and standard deviation), which carry the domain characteristics of the training data, can be properly manipulated to improve the generalization ability of deep learning models. Common methods often consider the feature statistics as deterministic values measured from the learned features and do not explicitly consider the uncertain statistics discrepancy caused by potential domain shifts during testing. In this paper, we improve the network generalization ability by modeling the uncertainty of domain shifts with synthesized feature statistics during training. Specifically, we hypothesize that the feature statistic, after considering the potential uncertainties, follows a multivariate Gaussian distribution. Hence, each feature statistic is no longer a deterministic value, but a probabilistic point with diverse distribution possibilities. With the uncertain feature statistics, the models can be trained to alleviate the domain perturbations and achieve better robustness against potential domain shifts. Our method can be readily integrated into networks without additional parameters. Extensive experiments demonstrate that our proposed method consistently improves the network generalization ability on multiple vision tasks, including image classification, semantic segmentation, and instance retrieval. The code will be released soon at https://github.com/lixiaotong97/DSU.

show abstract

Section: Invariant Representation Learningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Uncertainty Modeling for Out-of-Distribution Generalization

Li¹,

Dai²,

Ge³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The aim of DG methods [25], [26], [27], [28] is to improve DNN performance in an unknown target domain using data from (several different) source domains. For semantic segmentation, several approaches have been proposed [25], [14], [29], e.g., Yue et al [14] mix the style of synthetic images with real images, using auxiliary source domain datasets, thereby learning more domain-invariant features. While our CBNA method for continual source-free UDA is applied after pre-training and using only the pre-trained model and target domain data, DG is applied during pre-training on source data (usually with labels) and without target data.…”

Section: A Domain Generalization (Dg)mentioning

confidence: 99%

Continual BatchNorm Adaptation (CBNA) for Semantic Segmentation

Klingner¹,

Ayache²,

Fingscheidt³

2022

Preprint

View full text Add to dashboard Cite

Environment perception in autonomous driving vehicles often heavily relies on deep neural networks (DNNs), which are subject to domain shifts, leading to a significantly decreased performance during DNN deployment. Usually, this problem is addressed by unsupervised domain adaptation (UDA) approaches trained either simultaneously on source and target domain datasets or even source-free only on target data in an offline fashion. In this work, we further expand a source-free UDA approach to a continual and therefore online-capable UDA on a single-image basis for semantic segmentation. Accordingly, our method only requires the pre-trained model from the supplier (trained in the source domain) and the current (unlabeled target domain) camera image. Our method Continual BatchNorm Adaptation (CBNA) modifies the source domain statistics in the batch normalization layers, using target domain images in an unsupervised fashion, which yields consistent performance improvements during inference. Thereby, in contrast to existing works, our approach can be applied to improve a DNN continuously on a single-image basis during deployment without access to source data, without algorithmic delay, and nearly without computational overhead. We show the consistent effectiveness of our method across a wide variety of source/target domain settings for semantic segmentation. As part of this work, our code will be made publicly available. 1

show abstract

“…To reduce the large gap between synthetic data and the real-world data, DG methods usually augment the synthetic samples [38,12] with the styles of ImageNet [7] or conditionally align the outputs [2,3] between the segmentation model and the ImageNet pre-trained model. On the other hand, some works are proposed to learn domain-invariant features by removing domain-specific information [5,27] or feature augmentation [32]. Recently, Liu et al [22] Source-Free Domain Adaptation.…”

Section: Related Workmentioning

confidence: 99%

Source-Free Open Compound Domain Adaptation in Semantic Segmentation

Zhao¹,

Zhang²,

Luo³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this work, we introduce a new concept, named source-free open compound domain adaptation (SF-OCDA), and study it in semantic segmentation. SF-OCDA is more challenging than the traditional domain adaptation but it is more practical. It jointly considers (1) the issues of data privacy and data storage and ( 2) the scenario of multiple target domains and unseen open domains. In SF-OCDA, only the source pre-trained model and the target data are available to learn the target model. The model is evaluated on the samples from the target and unseen open domains. To solve this problem, we present an effective framework by separating the training process into two stages: (1) pre-training a generalized source model and (2) adapting a target model with self-supervised learning. In our framework, we propose the Cross-Patch Style Swap (CPSS) to diversify samples with various patch styles in the feature-level, which can benefit the training of both stages. First, CPSS can significantly improve the generalization ability of the source model, providing more accurate pseudo-labels for the latter stage. Second, CPSS can reduce the influence of noisy pseudo-labels and also avoid the model overfitting to the target domain during self-supervised learning, consistently boosting the performance on the target and open domains. Experiments demonstrate that our method produces state-of-the-art results on the C-Driving dataset. Furthermore, our model also achieves the leading performance on CityScapes for domain generalization. * Equal contribution. Preprint. Under review.

show abstract

RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening

Cited by 179 publications

References 56 publications

Uncertainty Modeling for Out-of-Distribution Generalization

Uncertainty Modeling for Out-of-Distribution Generalization

Continual BatchNorm Adaptation (CBNA) for Semantic Segmentation

Source-Free Open Compound Domain Adaptation in Semantic Segmentation

Contact Info

Product

Resources

About