SEMI2I: Semantically Consistent Image-to-Image Translation for Domain Adaptation of Remote Sensing Data

Taşar, Onur; Happy, S L; Tarabalka, Yuliya; Alliez, Pierre

doi:10.1109/igarss39084.2020.9323711

Cited by 29 publications

(24 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Considering this variation, the above assumptions often do not hold in remote sensing. There are few works related to domain adaptation [27], [28], [29] that try to align the target distribution with the source distribution. However, such methods are only effective when the domain shift between the source and target is small.…”

Section: Introductionmentioning

confidence: 99%

An Advanced Dirichlet Prior Network for Out-of-Distribution Detection in Remote Sensing

Gawlikowski

Saha

Kruspe

et al. 2022

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

Remote sensing deals with a plethora of sensors, a large number of classes/categories, and a huge variation in geography. Owing to the difficulty of collecting labeled data uniformly representing all scenarios, data-hungry deep learning models are often trained with labeled data in a source domain that is limited in the above-mentioned aspects. However during test/inference phase, such deep learning models are often subjected to a distributional shift, also called out-of-distribution (OOD) samples, in the form of unseen classes, geographic differences, and multi-sensor differences. Deep learning models can behave in an unexpected manner when subjected to such distributional uncertainties. Vulnerability to OOD data severely reduces the reliability of deep learning models and trusting on such predictions in absence of any reliability indicator may lead to wrong policy decisions or mishaps in time-bound remote sensing applications. Motivated by this, in this work, we propose a Dirichlet Prior Network-based model to quantify distributional uncertainty of deep learning-based remote sensing models. The approach seeks to maximize the representation gap between the in-domain and OOD examples for better segregation of OOD samples at test time. Extensive experiments on several remote sensing image classification data sets demonstrate that the proposed model can quantify distributional uncertainty. To the best of our knowledge this is the first work to elaborately study distributional uncertainty in context of remote sensing. The codes are publicly available at https://gitlab.lrz.de/ai4eo/Uncertainty/-/ tree/main/DPN-RS.

show abstract

Section: Introductionmentioning

confidence: 99%

An Advanced Dirichlet Prior Network for Out-of-Distribution Detection in Remote Sensing

Gawlikowski

Saha

Kruspe

et al. 2022

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

show abstract

“…In computer vision, different strategies are adopted for domain adaptation, e.g., adversarial training [11], [12] and discrepancies measurement minimization [13]. There are several problem settings according to the alignment of label space of domains, e.g., open-set, close-set, and universal domain adaptation [14], or the number of sources and targets, e.g., single-source-single-target, multi-source [15], [16], multitarget domain adaptation [17], [18].…”

Section: Related Workmentioning

confidence: 99%

Multitarget Domain Adaptation for Remote Sensing Classification Using Graph Neural Network

Saha

Shu

Zhu

2022

IEEE Geosci. Remote Sensing Lett.

View full text Add to dashboard Cite

Remote sensing deals with huge variations in geography, acquisition season, and a plethora of sensors. Considering the difficulty of collecting labeled data uniformly representing all scenarios, data-hungry deep learning models are often trained with labeled data in a source domain that is limited in the abovementioned aspects. Domain adaptation methods can adapt such model for applying on target domains with different distributions from the source domain. However, most remote sensing domain adaptation methods are designed for single-target, thus requiring a separate target classifier to be trained for each target domain.To mitigate this, we propose multi-target domain adaptation in which a single classifier is learned for multiple unlabeled target domains. To build a multi-target classifier, it may be beneficial to effectively aggregate features from the labeled source and different unlabeled target domains. Towards this, we exploit coteaching based on the graph neural network that is capable of leveraging unlabeled data. We use a sequential adaptation strategy that first adapts on the easier target domains assuming that the network finds it easier to adapt to the closest target domain. We validate the proposed method on two different datasets, representing geographical and seasonal variation. Code is available at https://gitlab.lrz.de/ai4eo/da-multitarget-gnn/

show abstract

“…proposed ColorMapGAN [48], SemI2I [49] and DAugNet [50] to perform image-to-image translation between satellite image pairs to reduces the impact of domain gap. All the above mentioned methods focus on adapting the source segmentation model to the target domain without taking into account the opposite target-to-source direction that is beneficial.…”

Section: Related Work 21 Domain Adaptationmentioning

confidence: 99%

BiFDANet: Unsupervised Bidirectional Domain Adaptation for Semantic Segmentation of Remote Sensing Images

et al. 2022

View full text Add to dashboard Cite

When segmenting massive amounts of remote sensing images collected from different satellites or geographic locations (cities), the pre-trained deep learning models cannot always output satisfactory predictions. To deal with this issue, domain adaptation has been widely utilized to enhance the generalization abilities of the segmentation models. Most of the existing domain adaptation methods, which based on image-to-image translation, firstly transfer the source images to the pseudo-target images, adapt the classifier from the source domain to the target domain. However, these unidirectional methods suffer from the following two limitations: (1) they do not consider the inverse procedure and they cannot fully take advantage of the information from the other domain, which is also beneficial, as confirmed by our experiments; (2) these methods may fail in the cases where transferring the source images to the pseudo-target images is difficult. In this paper, in order to solve these problems, we propose a novel framework BiFDANet for unsupervised bidirectional domain adaptation in the semantic segmentation of remote sensing images. It optimizes the segmentation models in two opposite directions. In the source-to-target direction, BiFDANet learns to transfer the source images to the pseudo-target images and adapts the classifier to the target domain. In the opposite direction, BiFDANet transfers the target images to the pseudo-source images and optimizes the source classifier. At test stage, we make the best of the source classifier and the target classifier, which complement each other with a simple linear combination method, further improving the performance of our BiFDANet. Furthermore, we propose a new bidirectional semantic consistency loss for our BiFDANet to maintain the semantic consistency during the bidirectional image-to-image translation process. The experiments on two datasets including satellite images and aerial images demonstrate the superiority of our method against existing unidirectional methods.

show abstract

SEMI2I: Semantically Consistent Image-to-Image Translation for Domain Adaptation of Remote Sensing Data

Cited by 29 publications

References 8 publications

An Advanced Dirichlet Prior Network for Out-of-Distribution Detection in Remote Sensing

An Advanced Dirichlet Prior Network for Out-of-Distribution Detection in Remote Sensing

Multitarget Domain Adaptation for Remote Sensing Classification Using Graph Neural Network

BiFDANet: Unsupervised Bidirectional Domain Adaptation for Semantic Segmentation of Remote Sensing Images

Contact Info

Product

Resources

About