SAR-to-Optical Image Translation Using Supervised Cycle-Consistent Adversarial Networks

Wang, Lei; Xu, Xin; Yu, Yue; Yang, Rui; Gui, Rong; Xu, Zhaozhuo; Pu, Fangling

doi:10.1109/access.2019.2939649

Cited by 106 publications

(68 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CycleGAN has a two-step transformation of source domain image -first by trying to map it to target domain and then map back to the source domain and aims to reduce the gap between the reconstructed image and the original image. • S-CycleGAN [35] It is a very recent solution that targets for remote sensing image conversion. It leverage the cycle loss similar to CycleGAN and MSE loss similar to Pix2Pix to achieve an enhanced image to image translation.…”

Section: Data Statisticsmentioning

confidence: 99%

An Enhanced GAN Model for Automatic Satellite-to-Map Image Conversion

et al. 2020

View full text Add to dashboard Cite

Location-based service significantly relies on accurate and up-to-date maps. The conventional map generation involves labor-intensive and time-consuming manual efforts, which restricts the map-update frequency to a few years or even longer. In recent years, satellite images become more ubiquitous, and converting them to map-style images has attracted attention due to its frequent-updating and cost-effective nature. Generative adversarial network (GAN) is a promising approach for automatic satellite-to-map image conversion. However, it is still challenging to process satellite images when the underlying road structure is complex and irregular, or when some objects are visually indistinguishable due to obstruction or bad weather. To address these issues, we propose an enhanced GAN model to generate improved quality map images by bringing in the external geographic data as implicit guidance. The textual geographic data is converted to an image so that it can work collaboratively and seamlessly with the satellite image during the conversion. A high-level semantic regulation is also introduced to further reduce the noisy patterns generated during the translation, which occur frequently for the regions with sparse geographic data. The proposed method is versatile to various backbone GAN structures with a 20% performance improvement on three popular metrics (Inception Score, Frechet Inception Distance Score and SSIM score). Our proposed Semantic-regulated Geographic GAN (SG-GAN) is anticipated to reduce the manual identification efforts in broad geospatial applications.

show abstract

Section: Data Statisticsmentioning

confidence: 99%

An Enhanced GAN Model for Automatic Satellite-to-Map Image Conversion

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Engin et al [44] proposed an enhanced CycleGAN by com- bining cycle consistency loss and perceptual loss to generate haze-free images. Wang et al [45] translated the SAR patches to optical image patches. Gorti and Ma [46] used the Cycle-GAN to generate images from text.…”

Section: B Cycle-consistent Adversarial Networkmentioning

confidence: 99%

Enhanced Text Matching Based on Semantic Transformation

et al. 2020

View full text Add to dashboard Cite

Text matching is the core of natural language processing (NLP) system. It's considered as a touchstone of the NLP, and it aims to find whether text pairs are equal in semantics. However, the semantic gap in text matching is still an open problem to solve. Inspired by successes of cycle-consistent adversarial network (CycleGAN) in image domain transformation, we propose an enhanced text matching method based on the CycleGAN combined with the Transformer network. Based on the proposed method, the text semantics in a source domain is transferred to a similar or different target domain, and the semantic distance between text pairs is decreased. Meanwhile, we demonstrate our method in paraphrase identification and question answer matching. The matching degree is computed by a standard text matching model to evaluate the transforming influence on narrowing the text semantic gap. The experiments show that our method achieves text domain adaptation, and the effects on different matching models are remarkable. INDEX TERMS Cycle-consistent adversarial network, domain adaptation, semantic transformation, text matching, transformer network.

show abstract

“…cGANs have attracted considerable interest in the remote sensing community [41], as they allow to generate desired artificial data based on a specified target output and have achieved promising results in many fields, such as image inpainting [42][43][44], image manipulation [45][46][47], and image translation [48][49][50][51][52]. More specifically, cGANs can be employed to efficiently translate SAR images to optical images, and have been proved to be suitable in the SAR-to-optical translation process [6,16,17,20,[53][54][55][56][57][58]. There are several cGANs-based SAR-to-optical image translation methods.…”

Section: Introductionmentioning

confidence: 99%

Comparative Analysis of Edge Information and Polarization on SAR-to-Optical Translation Based on Conditional Generative Adversarial Networks

Zhang

Liu

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

To accurately describe dynamic vegetation changes, high temporal and spectral resolution data are urgently required. Optical images contain rich spectral information but are limited by poor weather conditions and cloud contamination. Conversely, synthetic-aperture radar (SAR) is effective under all weather conditions but contains insufficient spectral information to recognize certain vegetation changes. Conditional adversarial networks (cGANs) can be adopted to transform SAR images (Sentinel-1) into optical images (Landsat8), which exploits the advantages of both optical and SAR images. As the features of SAR and optical remote sensing data play a decisive role in the translation process, this study explores the quantitative impact of edge information and polarization (VV, VH, VV&VH) on the peak signal-to-noise ratio, structural similarity index measure, correlation coefficient (r), and root mean squared error. The addition of edge information improves the structural similarity between generated and real images. Moreover, using the VH and VV&VH polarization modes as the input provides the cGANs with more effective information and results in better image quality. The optimal polarization mode with the addition of edge information is VV&VH, whereas that without edge information is VV. Near-infrared and short-wave infrared bands in the generated image exhibit higher accuracy (r > 0.8) than visible light bands. The conclusions of this study could serve as an important reference for selecting cGANs input features, and as a potential reference for the applications of cGANs to the SAR-to-optical translation of other multi-source remote sensing data.

show abstract

SAR-to-Optical Image Translation Using Supervised Cycle-Consistent Adversarial Networks

Cited by 106 publications

References 51 publications

An Enhanced GAN Model for Automatic Satellite-to-Map Image Conversion

An Enhanced GAN Model for Automatic Satellite-to-Map Image Conversion

Enhanced Text Matching Based on Semantic Transformation

Comparative Analysis of Edge Information and Polarization on SAR-to-Optical Translation Based on Conditional Generative Adversarial Networks

Contact Info

Product

Resources

About