Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentation

Zhao, Yuyang; Zhong, Zhun; Zhao, Na; Sebe, Nicu; Lee, Gim Hee

doi:10.1007/978-3-031-19815-1_31

Cited by 32 publications

(24 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Augmenting the style distribution for narrowing the domain shift issue has been explored in domain generation [34,60,66], image segmentation [4,63], and CD-FSL [11]. Concretely, MixStyle [66], AdvStyle [63], DSU [34], and wave-SAN [11] synthesize styles without extra parameters via mixing, attacking, sampling from a Gaussian distribution, and swapping. MaxStyle [4] and L2D [60] require additional network modules and complex auxiliary tasks to help generate the new styles.…”

Section: Related Workmentioning

confidence: 99%

“…MaxStyle [4] and L2D [60] require additional network modules and complex auxiliary tasks to help generate the new styles. Typically, AdvStyle [63] is the most related work to us. Thus, we highlight the key differences: 1) AdvStyle attacks styles on the image, while we attack styles on multiple feature spaces with a progressive attacking method; 2) AdvStyle uses the same task loss (segmentation) for attacking and optimization; in contrast, we use the classical classification loss to attack the styles, while utilize the task loss (FSL) to optimize the whole network.…”

Section: Related Workmentioning

confidence: 99%

“…Methods? To show the advantages of our Style-FGSM against other style augmentation methods, we introduce several competitors including "StyleGaus", MixStyle [66], Ad-vStyle [63], and DSU [34]. Typically, "StyleGaus" that adds random Gaussian noises into the styles is introduced as a simple but reasonable baseline.…”

Section: Is Style-fgsm Better Than Other Style Augmentationmentioning

confidence: 99%

“…Typically, "StyleGaus" that adds random Gaussian noises into the styles is introduced as a simple but reasonable baseline. MixStyle [66], AdvStyle [63], and DSU [34] which are initially designed for other tasks, e.g., segmentation and domain generation are also adapted.…”

Section: Is Style-fgsm Better Than Other Style Augmentationmentioning

confidence: 99%

See 3 more Smart Citations

Meta Style Adversarial Training for Cross-Domain Few-Shot Learning

Fu¹,

Xie²,

Fu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Cross-Domain Few-Shot Learning (CD-FSL) is a recently emerging task that tackles few-shot learning across different domains. It aims at transferring prior knowledge learned on the source dataset to novel target datasets. The CD-FSL task is especially challenged by the huge domain gap between different datasets. Critically, such a domain gap actually comes from the changes of visual styles, and wave-SAN [11] empirically shows that spanning the style distribution of the source data helps alleviate this issue. However, wave-SAN simply swaps styles of two images. Such a vanilla operation makes the generated styles "real" and "easy", which still fall into the original set of the source styles. Thus, inspired by vanilla adversarial learning, a novel model-agnostic meta Style Adversarial training (StyleAdv) method together with a novel style adversarial attack method is proposed for CD-FSL. Particularly, our style attack method synthesizes both "virtual" and "hard" adversarial styles for model training. This is achieved by perturbing the original style with the signed style gradients. By continually attacking styles and forcing the model to recognize these challenging adversarial styles, our model is gradually robust to the visual styles, thus boosting the generalization ability for novel target datasets. Besides the typical CNN-based backbone, we also employ our StyleAdv method on large-scale pretrained vision transformer. Extensive experiments conducted on eight various target datasets show the effectiveness of our method. Whether built upon ResNet or ViT, we achieve the new state of the art for CD-FSL. Codes and models will be released.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Is Style-fgsm Better Than Other Style Augmentationmentioning

confidence: 99%

Section: Is Style-fgsm Better Than Other Style Augmentationmentioning

confidence: 99%

See 2 more Smart Citations

Meta Style Adversarial Training for Cross-Domain Few-Shot Learning

Fu¹,

Xie²,

Fu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…However, the domain shift is usually agnostic in real-world scenarios since the target data is not available for training. This issue inspires the research area of domain generalization (DG) [1,22,27,28,30,34,41,43,45,47,51,52,54,74,75,[78][79][80], which is aimed to make models trained on seen domains achieve accurate predictions on unseen domainss, i.e., the conditional distribution P (Y |X) is robust with shifted marginal distribution P (X). Canonical DG focuses on learning a domaininvariant feature distribution P (F (X)) across domains for the robustness of conditional distribution P (Y |F (X)).…”

Section: Introductionmentioning

confidence: 99%

Attention Diversification for Domain Generalization

Meng

Chen

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) have demonstrated gratifying results at learning discriminative features. However, when applied to unseen domains, state-of-the-art models are usually prone to errors due to domain shift. After investigating this issue from the perspective of shortcut learning, we find the devils lie in the fact that models trained on different domains merely bias to different domain-specific features yet overlook diverse task-related features. Under this guidance, a novel Attention Diversification framework is proposed, in which Intra-Model and Inter-Model Attention Diversification Regularization are collaborated to reassign appropriate attention to diverse task-related features. Briefly, Intra-Model Attention Diversification Regularization is equipped on the high-level feature maps to achieve in-channel discrimination and cross-channel diversification via forcing different channels to pay their most salient attention to different spatial locations. Besides, Inter-Model Attention Diversification Regularization is proposed to further provide task-related attention diversification and domain-related attention suppression, which is a paradigm of "simulate, divide and assemble": simulate domain shift via exploiting multiple domain-specific models, divide attention maps into task-related and domain-related groups, and assemble them within each group respectively to execute regularization. Extensive experiments and analyses are conducted on various benchmarks to demonstrate that our method achieves state-of-the-art performance over other competing methods. Code is available at https: //github.com/hikvision-research/DomainGeneralization.

show abstract

Per-class curriculum for Unsupervised Domain Adaptation in semantic segmentation

Alcover-Couso,

SanMiguel,

Escudero-Viñolo

et al. 2024

Vis Comput

View full text Add to dashboard Cite

Accurate training of deep neural networks for semantic segmentation requires a large number of pixel-level annotations of real images, which are expensive to generate or not even available. In this context, Unsupervised Domain Adaptation (UDA) can transfer knowledge from unlimited synthetic annotations to unlabeled real images of a given domain. UDA methods are composed of an initial training stage with labeled synthetic data followed by a second stage for feature alignment between labeled synthetic and unlabeled real data. In this paper, we propose a novel approach for UDA focusing the initial training stage, which leads to increased performance after adaptation. We introduce a curriculum strategy where each semantic class is learned progressively. Thereby, better features are obtained for the second stage. This curriculum is based on: (1) a class-scoring function to determine the difficulty of each semantic class, (2) a strategy for incremental learning based on scoring and pacing functions that limits the required training time unlike standard curriculum-based training and (3) a training loss to operate at class level. We extensively evaluate our approach as the first stage of several state-of-the-art UDA methods for semantic segmentation. Our results demonstrate significant performance enhancements across all methods: improvements of up to 10% for entropy-based techniques and 8% for adversarial methods. These findings underscore the dependency of UDA on the accuracy of the initial training. The implementation is available at https://github.com/vpulab/PCCL.

show abstract

Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentation

Cited by 32 publications

References 33 publications

Meta Style Adversarial Training for Cross-Domain Few-Shot Learning

Meta Style Adversarial Training for Cross-Domain Few-Shot Learning

Attention Diversification for Domain Generalization

Per-class curriculum for Unsupervised Domain Adaptation in semantic segmentation

Contact Info

Product

Resources

About