DAugNet: Unsupervised, Multisource, Multitarget, and Life-Long Domain Adaptation for Semantic Segmentation of Satellite Images

Taşar, Onur; Giros, Alain; Tarabalka, Yuliya; Alliez, Pierre; Clerc, Sébastien

doi:10.1109/tgrs.2020.3006161

Cited by 60 publications

(41 citation statements)

References 49 publications

(72 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Whereas in RS images, the content of different parts of the images may come with a large variation and thus are completely unstructured, in addition to which the atmospheric effects create even larger variations on the object appearances, let alone the drastic change of land patterns across the different geographical region (e.g., urban vs. suburban, tropical area vs. frigid area, etc.). It is well-noted that every single RS image could be a domain [34,35]. Therefore, to scale up classification capabilities, transferability issues remain one of the main challenges to face.…”

Section: Model and Scene Transferabilitymentioning

confidence: 99%

A Review of Landcover Classification with Very-High Resolution Remotely Sensed Optical Images—Analysis Unit, Model Scalability and Transferability

Qin

Liu

2022

Remote Sensing

View full text Add to dashboard Cite

As an important application in remote sensing, landcover classification remains one of the most challenging tasks in very-high-resolution (VHR) image analysis. As the rapidly increasing number of Deep Learning (DL) based landcover methods and training strategies are claimed to be the state-of-the-art, the already fragmented technical landscape of landcover mapping methods has been further complicated. Although there exists a plethora of literature review work attempting to guide researchers in making an informed choice of landcover mapping methods, the articles either focus on the review of applications in a specific area or revolve around general deep learning models, which lack a systematic view of the ever advancing landcover mapping methods. In addition, issues related to training samples and model transferability have become more critical than ever in an era dominated by data-driven approaches, but these issues were addressed to a lesser extent in previous review articles regarding remote sensing classification. Therefore, in this paper, we present a systematic overview of existing methods by starting from learning methods and varying basic analysis units for landcover mapping tasks, to challenges and solutions on three aspects of scalability and transferability with a remote sensing classification focus including (1) sparsity and imbalance of data; (2) domain gaps across different geographical regions; and (3) multi-source and multi-view fusion. We discuss in detail each of these categorical methods and draw concluding remarks in these developments and recommend potential directions for the continued endeavor.

show abstract

Section: Model and Scene Transferabilitymentioning

confidence: 99%

A Review of Landcover Classification with Very-High Resolution Remotely Sensed Optical Images—Analysis Unit, Model Scalability and Transferability

Qin

Liu

2022

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…proposed ColorMapGAN [48], SemI2I [49] and DAugNet [50] to perform image-to-image translation between satellite image pairs to reduces the impact of domain gap. All the above mentioned methods focus on adapting the source segmentation model to the target domain without taking into account the opposite target-to-source direction that is beneficial.…”

Section: Related Work 21 Domain Adaptationmentioning

confidence: 99%

BiFDANet: Unsupervised Bidirectional Domain Adaptation for Semantic Segmentation of Remote Sensing Images

et al. 2022

View full text Add to dashboard Cite

When segmenting massive amounts of remote sensing images collected from different satellites or geographic locations (cities), the pre-trained deep learning models cannot always output satisfactory predictions. To deal with this issue, domain adaptation has been widely utilized to enhance the generalization abilities of the segmentation models. Most of the existing domain adaptation methods, which based on image-to-image translation, firstly transfer the source images to the pseudo-target images, adapt the classifier from the source domain to the target domain. However, these unidirectional methods suffer from the following two limitations: (1) they do not consider the inverse procedure and they cannot fully take advantage of the information from the other domain, which is also beneficial, as confirmed by our experiments; (2) these methods may fail in the cases where transferring the source images to the pseudo-target images is difficult. In this paper, in order to solve these problems, we propose a novel framework BiFDANet for unsupervised bidirectional domain adaptation in the semantic segmentation of remote sensing images. It optimizes the segmentation models in two opposite directions. In the source-to-target direction, BiFDANet learns to transfer the source images to the pseudo-target images and adapts the classifier to the target domain. In the opposite direction, BiFDANet transfers the target images to the pseudo-source images and optimizes the source classifier. At test stage, we make the best of the source classifier and the target classifier, which complement each other with a simple linear combination method, further improving the performance of our BiFDANet. Furthermore, we propose a new bidirectional semantic consistency loss for our BiFDANet to maintain the semantic consistency during the bidirectional image-to-image translation process. The experiments on two datasets including satellite images and aerial images demonstrate the superiority of our method against existing unidirectional methods.

show abstract

“…Lastly, the estimated target domain parameters are weakly transferable to source domain and therefore diminishes re-usability. Motivated by the continuous proliferation of satellite imagery and an increase in multiple sources and target domains having different data distributions, the DAugNet is proposed in [26] to extend adversarial-based unsupervised learning methods toward multisource, multitarget, and life-long domain adaptation problems.…”

Section: Related Workmentioning

confidence: 99%

ReSFlow: A Remote Sensing Imagery Data-Flow for Improved Model Generalization

Lunga

Arndt

Gerrand

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

As satellite imagery collections continue to grow at an astonishing rate, so is the demand for automated and scalable object detection and segmentation. Scaling computational activities demand models that generalize well across various challenges that can hamper progress, including 1) diverse imaging and geographic conditions, 2) sampling bias in training data, 3) manual ground truth collection, 4) tooling for model reuse and accountability assessment, and 5) poor model training strategies. A great deal of progress has been made on these challenges. We contribute to the improvement through further development of ReSFlow, a workflow that breaks the problem of model generalization into a collection of specialized exploitations. ReSFlow partitions imagery collections into homogeneous buckets equipped with exploitation models trained to perform well under each bucket's specific context. Essentially ReSFlow aims for generalization through stratification. Therefore, within a bucket, exploitation is a homogeneous process that mitigates heterogeneity challenges, including the number of training data and data biases that can occur over varied conditions. Furthermore, custom model architectures and rich training strategies effective for within bucket conditions can be developed. Meanwhile, across bucket performance metrics support systematic views of the workflow leading to optimal data labeling allocations and indications that further specialization (and retooling of models) is warranted. Herein, we discuss the formation of models during the framework's "Offline Initialization" stage. Lastly, we exploit the inherent parallelism due to bucketing to introduce model reuse and demonstrate efficacy by reducing an 89-day manual data labeling cost to zero-days in a new area of interest.

show abstract

DAugNet: Unsupervised, Multisource, Multitarget, and Life-Long Domain Adaptation for Semantic Segmentation of Satellite Images

Cited by 60 publications

References 49 publications

A Review of Landcover Classification with Very-High Resolution Remotely Sensed Optical Images—Analysis Unit, Model Scalability and Transferability

A Review of Landcover Classification with Very-High Resolution Remotely Sensed Optical Images—Analysis Unit, Model Scalability and Transferability

BiFDANet: Unsupervised Bidirectional Domain Adaptation for Semantic Segmentation of Remote Sensing Images

ReSFlow: A Remote Sensing Imagery Data-Flow for Improved Model Generalization

Contact Info

Product

Resources

About