Continual Unsupervised Domain Adaptation for Semantic Segmentation using a Class-Specific Transfer

Marsden, Robert; Wiewel, Felix; Döbler, Mario; Yang, Yang; Yang, Bin

doi:10.1109/ijcnn55064.2022.9892200

Cited by 11 publications

(7 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another line of work has explored diverse setups in terms of domain availability. Some propose to handle multiple source [58], [59] or target [17], [18], [19], [60], [61], [62], [63] domains. This can involve a single adaptation phase [58], [59], or multiple phases where different domains are experienced in different learning steps in an incremental fashion (but still with a fixed class set) [17], [18], [19], [62], [63], in fact, undertaking continual learning under the domain adaptation perspective.…”

Section: Related Workmentioning

confidence: 99%

Learning with Style: Continual Semantic Segmentation Across Tasks and Domains

Toldo,

Michieli,

Zanuttigh

2024

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Deep learning models dealing with image understanding in real-world settings must be able to adapt to a wide variety of tasks across different domains. Domain adaptation and class incremental learning deal with domain and task variability separately, whereas their unified solution is still an open problem. We tackle both facets of the problem together, taking into account the semantic shift within both input and label spaces. We start by formally introducing continual learning under task and domain shift. Then, we address the proposed setup by using style transfer techniques to extend knowledge across domains when learning incremental tasks and a robust distillation framework to effectively recollect task knowledge under incremental domain shift. The devised framework (LwS, Learning with Style) is able to generalize incrementally acquired task knowledge across all the domains encountered, proving to be robust against catastrophic forgetting. Extensive experimental evaluation on multiple autonomous driving datasets shows how the proposed method outperforms existing approaches, which prove to be ill-equipped to deal with continual semantic segmentation under both task and domain shift. The code is available at https://lttm.dei.unipd.it/paper data/LwS.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning with Style: Continual Semantic Segmentation Across Tasks and Domains

Toldo,

Michieli,

Zanuttigh

2024

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…Most methods in this category store information about a specific domain's style in order to transform source images into the styles of the target domains during training. Recent work achieves this by storing low-frequency components of the images for every domain [14], by capturing the style of the domain with generative models [15], [16] or by using a domain-specific memory to mitigate forgetting [17]. Supervised domain-incremental learning is rarely considered [11].…”

Section: A Continual Semantic Segmentationmentioning

confidence: 99%

“…CDA has the goal of adapting a model that is trained on a supervised source dataset to a sequence of different domains, for which no labels are provided. However, in order to compensate for the missing labels of the target domains, the model has access to the initial source dataset throughout the entire training sequence [36]. Methods in this category work mostly by storing information about the style of the specific domains, so that during training the source images can be transferred into the styles of the different target domains.…”

Section: Continual Unsupervised Domain Adaptationmentioning

confidence: 99%

“…Methods in this category work mostly by storing information about the style of the specific domains, so that during training the source images can be transferred into the styles of the different target domains. This can be achieved by storing low-frequency components of the domains [53] or by capturing the style using generative models [36,60]. Other recent work proposes to use a target-specific memory for every domain to mitigate forgetting [28].…”

Section: Continual Unsupervised Domain Adaptationmentioning

confidence: 99%

“…This phenomenon is usually referred to as catastrophic forgetting and is a fundamental challenge when training a neural network on a continuous stream of data. Recently proposed methods mostly mitigate this challenge by replaying data from previous domains, re-estimating statistics or even in an unsupervised manner by transferring training images in the style of the novel domain [36,53,60]. The focus of our work is to study how the internal representations of se-mantic segmentation models are affected during domainincremental learning and how efficient feature reuse can mitigate forgetting without explicit replay of the previous domain.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Causes of Catastrophic Forgetting in Class-Incremental Semantic Segmentation

Kalb¹,

Beyerer

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Research in the field of Continual Semantic Segmentation is mainly investigating novel learning algorithms to overcome catastrophic forgetting of neural networks. Most recent publications have focused on improving learning algorithms without distinguishing effects caused by the choice of neural architecture. Therefore, we study how the choice of neural network architecture affects catastrophic forgetting in class-and domainincremental semantic segmentation. Specifically, we compare the well-researched CNNs to recently proposed Transformers and Hybrid architectures, as well as the impact of the choice of novel normalization layers and different decoder heads. We find that traditional CNNs like ResNet have high plasticity but low stability, while transformer architectures are much more stable. When the inductive biases of CNN architectures are combined with transformers in hybrid architectures, it leads to higher plasticity and stability. The stability of these models can be explained by their ability to learn general features that are robust against distribution shifts. Experiments with different normalization layers show that Continual Normalization achieves the best trade-off in terms of adaptability and stability of the model. In the class-incremental setting, the choice of the normalization layer has much less impact. Our experiments suggest that the right choice of architecture can significantly reduce forgetting even with naive fine-tuning and confirm that for real-world applications, the architecture is an important factor in designing a continual learning model.

show abstract

The segmentation effect of style transfer on fetal head ultrasound image: a study of multi-source data

Zhou

Wang

et al. 2023

Med Biol Eng Comput

View full text Add to dashboard Cite

Continual Unsupervised Domain Adaptation for Semantic Segmentation using a Class-Specific Transfer

Cited by 11 publications

References 30 publications

Learning with Style: Continual Semantic Segmentation Across Tasks and Domains

Learning with Style: Continual Semantic Segmentation Across Tasks and Domains

Causes of Catastrophic Forgetting in Class-Incremental Semantic Segmentation

The segmentation effect of style transfer on fetal head ultrasound image: a study of multi-source data

Contact Info

Product

Resources

About