An Enhanced GAN Model for Automatic Satellite-to-Map Image Conversion

Abstract

Location-based service significantly relies on accurate and up-to-date maps. The conventional map generation involves labor-intensive and time-consuming manual efforts, which restricts the map-update frequency to a few years or even longer. In recent years, satellite images become more ubiquitous, and converting them to map-style images has attracted attention due to its frequent-updating and cost-effective nature. Generative adversarial network (GAN) is a promising approach for automatic satellite-to-map image conversion. However, it is still challenging to process satellite images when the underlying road structure is complex and irregular, or when some objects are visually indistinguishable due to obstruction or bad weather. To address these issues, we propose an enhanced GAN model to generate improved quality map images by bringing in the external geographic data as implicit guidance. The textual geographic data is converted to an image so that it can work collaboratively and seamlessly with the satellite image during the conversion. A high-level semantic regulation is also introduced to further reduce the noisy patterns generated during the translation, which occur frequently for the regions with sparse geographic data. The proposed method is versatile to various backbone GAN structures with a 20% performance improvement on three popular metrics (Inception Score, Frechet Inception Distance Score and SSIM score). Our proposed Semantic-regulated Geographic GAN (SG-GAN) is anticipated to reduce the manual identification efforts in broad geospatial applications.

GAN-Based Satellite Imaging: A Survey on Techniques and Applications

“…The unpaired t-test is run to investigate if the GANs can improve the performance compared to non-GAN approaches. For example, the experimental results reported in [42] for the satellite to map conversion allows us to create distinguished result populations to run unpaired t-tests. In this research, the segmentation-based methods and GAN-based methods were compared in terms of IS, FID, and SSIM.…”

“…Therefore, the solely image-based GAN framework is not sufficient for this specific satellite-to-map image conversion task. To overcome the above obstacles, Zhang et al [42] proposed an enhanced GAN model to generate improved-quality map images using the GPS coordinates as additional knowledge. In addition to circularity constraint, Song et al [44] integrated geometrical consistency constraint into the whole architecture to reduce the translation's semantic distortions.…”

“…PSNR, largely based on mean-squared error (MSE), is a metric that is based on the average difference between corresponding pixels in two images. • The realism and diversity of the synthesized images are measured by Inception Score (IS) [42], Top-k prediction accuracy, Frechet Inception Distance (FID) score [42] and KL divergence [59]. • The pixel-wise semantic consistency of the synthesized images is measured using mean Intersection-over-Union (mIoU).…”

Representing Vector Geographic Information As a Tensor for Deep Learning Based Map Generalisation

“…In the literature, most experiments of map generation with GAN are about generating an image of a map in the style of GoogleMap from the corresponding aerial photograph, and vice versa. Isola et al (2018) proposed a generic image-to-image translation model called Pix2Pix, which was later improved by many researchers to better deal with the generation of maps (Ganguli et al, 2019;Chen et al, 2020;Zhang et al, 2020;Li et al, 2020). In their work, the scale of the styletransferred map is similar to that of the aerial photograph.…”

Facade Style Mixing using Artificial Intelligence for Urban Infill

“…There have recently been several experiments on generative approaches, variational auto-encoders, in the area of computer vision (VAEs) [4] and generative adversarial nets (GANs) [5]. In specific, GANs display important results in different tasks of computer vision, such as image creation [6][7][8], image conversion [9], super-resolution [10] and text-to-image synthesis [11]. GANs have already been applied to facade generation [8].…”