Pix2pix Conditional Generative Adversarial Network with MLP Loss Function for Cloud Removal in a Cropland Time Series

Christovam, Luiz E.; Shimabukuro, Milton Hirokazu; Galo, Maria de Lourdes Bueno Trindade; Honkavaara, Eija

doi:10.3390/rs14010144

Cited by 17 publications

(8 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A PIX2PIX GAN's, architecture is made up of two basic components: a generator and a discriminator, as mentioned in the elaboration of GANs since it's a type of GAN algorithm. The generator is in charge of creating the output image, while the discriminator is in charge of assessing the realism of the created image and delivering input to the generator [70].…”

Section: B 2d Model Generationmentioning

confidence: 99%

PIXGAN-Drone: 3D Avatar of Human Body Reconstruction From Multi-View 2D Images

Salim Rasheed,

Jabberi,

Hamdani

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Research on training Generative Adversarial Networks (GANs) to create 3D human body avatars from 2D datasets is underway. Research done in this field shows promise and has paved the way for significant advancements in a variety of applications, including virtual reality, sports analysis, cinematography, surveillance, and cinematography. By avoiding obstacles and producing high-resolution, rich-information multi-view (RGB) images, drone active tracking combined with aerial photography sensors can eliminate occlusions and enable 3D avatar body reconstruction. Due to several issues, such as restricted perspective coverage, obvious occlusions, and texture disappearance, 3D avatar reconstruction techniques encounter training failures that cause distortions and feature loss in 3D reconstructed models. The new endto-end trainable deep neural network methodology PIXGAN-Drone is presented in this paper for the photorealistic 3D avatar of the human body reconstruction from multi-view images. It is based on the integration of active tracking drones equipped with aerial photography sensors (stable automatic circular motion system) into the Pix2Pix GANs training framework to generate high-resolution 2D models. Conditional image-toimage translation and dynamic aerial perspectives can be used to develop realistic and accurate 3D models. This research conducted experiments on multiple datasets to demonstrate the improved performance of our method over state-of-the-art methods for various metrics (Chamfer, P2S, and CED). The results showed that our 3D reconstructed human avatars were 0.0293, 0.0271, and 0.0232 on RenderPeople, 0.0133, 0.0136, 0.0050 on People Snapshot (indoor), 0.0154, 0.0101, 0.0063 on People Snapshot (outdoor), and 0.0316, 0.0275, 0.0216 on Custom data-drone (collected dataset).INDEX TERMS 3D human avatar, 3D reconstruction, PIX2PIXGAN, body model rendering, drone active tracking.

show abstract

Section: B 2d Model Generationmentioning

confidence: 99%

PIXGAN-Drone: 3D Avatar of Human Body Reconstruction From Multi-View 2D Images

Salim Rasheed,

Jabberi,

Hamdani

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The Pix2Pix, using the discrete digital numbers (DNs) of conventional satellite images that range from 0 to 255, has demonstrated remarkable translation from one image to another image in satellite remote sensing applications [34][35][36]. The D2D framework translates one original data to another using the normalization process as the pre-processing and denormalization as the post-processing, which converts between original satelliteobserved albedo or brightness temperature (TB) and a numerical array each other before and after adversarial learning.…”

Section: Introductionmentioning

confidence: 99%

Temporal Resolution Enhancement of COMS Satellite Using Geo-Kompsat-2A Satellite Through Data-to-Data Translation

Kim,

Ryu,

Han

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

This study introduces a data-to-data (D2D) translation approach that utilizes a conditional adversarial learning framework to generate hypothetical data at the Meteorological Imager (MI) sensor on the Communication, Ocean, and Meteorological Satellite (COMS). The proposed D2D model produces virtual 10-minute data from actual 30-minute data by exploiting the 10-minute temporal resolution (TR) of the advanced MI (AMI) on GEO-KOMPSAT-2A (GK2A) during the overlapping observation period of the two satellites. Specifically, the D2D model uses one visible (VIS) at 0.64 µm channel and four infrared (IR) channels (3.8, 6.9, 10.8, and 12.3 µm) from AMI of the actual 30minute data from April 2020 to April 2022 to train and test the model. Subsequently, the D2D model is applied to simulate hypothetical 10 min-TR COMS data using the 30-minute COMS observation data from September 2019 to March 2020 during the coexistence period of the two satellites. Regression-calibrated COMS data alleviated the spectral response function differences between MI and AMI sensors. The proposed D2D method exhibits excellent statistical performance, with an average root mean square error of 0.056 for the VIS channel and 3.237 K,

show abstract

“…However, such dependency on the input data of MTcGAN implies a need for extensive evaluation. To address this, most previous studies have aimed to utilize an optical image taken on a reference date that is as close as possible to the prediction date to ensure satisfactory prediction performance [17,38]. As the physical conditions of crops change rapidly during the crop growth period, it is essential to analyze the impact of various input cases, such as by considering different image acquisition dates and crop growth stages.…”

Section: Introductionmentioning

confidence: 99%

“…Periodic image acquisition over the area of interest can facilitate the utilization of additional temporal information for SAR-to-optical translation. For example, multi-temporal cGAN (MTcGAN) utilizes SAR and optical image pairs acquired on the same or a similar date (hereafter referred to as a reference date) as well as a single SAR image acquired on a prediction date for extracting temporal change information from multi-temporal images [17,38]. Due to its ability to integrate additional information from an optical image on the reference date, it performed better than conventional SAR-to-optical image translation methods.…”

Section: Introductionmentioning

confidence: 99%

Assessing the Potential of Multi-Temporal Conditional Generative Adversarial Networks in SAR-to-Optical Image Translation for Early-Stage Crop Monitoring

Kwak,

Park

2024

Remote Sensing

View full text Add to dashboard Cite

The incomplete construction of optical image time series caused by cloud contamination is one of the major limitations facing the application of optical satellite images in crop monitoring. Thus, the construction of a complete optical image time series via image reconstruction of cloud-contaminated regions is essential for thematic mapping in croplands. This study investigates the potential of multi-temporal conditional generative adversarial networks (MTcGANs) that use a single synthetic aperture radar (SAR) image acquired on a prediction date and a pair of SAR and optical images acquired on a reference date in the context of early-stage crop monitoring. MTcGAN has an advantage over conventional SAR-to-optical image translation methods as it allows input data of various compositions. As the prediction performance of MTcGAN depends on the input data composition, the variations in the prediction performance should be assessed for different input data combination cases. Such an assessment was performed through experiments using Sentinel-1 and -2 images acquired in the US Corn Belt. MTcGAN outperformed existing SAR-to-optical image translation methods, including Pix2Pix and supervised CycleGAN (S-CycleGAN), in cases representing various input compositions. In particular, MTcGAN was substantially superior when there was little change in crop vitality between the reference and prediction dates. For the SWIR1 band, the root mean square error of MTcGAN (0.021) for corn was significantly improved by 54.4% and 50.0% compared to Pix2Pix (0.046) and S-CycleGAN (0.042), respectively. Even when there were large changes in crop vitality, the prediction accuracy of MTcGAN was more than twice that of Pix2Pix and S-CycleGAN. Without considering the temporal intervals between input image acquisition dates, MTcGAN was found to be beneficial when crops were visually distinct in both SAR and optical images. These experimental results demonstrate the potential of MTcGAN in SAR-to-optical image translation for crop monitoring during the early growth stage and can serve as a guideline for selecting appropriate input images for MTcGAN.

show abstract

Pix2pix Conditional Generative Adversarial Network with MLP Loss Function for Cloud Removal in a Cropland Time Series

Cited by 17 publications

References 37 publications

PIXGAN-Drone: 3D Avatar of Human Body Reconstruction From Multi-View 2D Images

PIXGAN-Drone: 3D Avatar of Human Body Reconstruction From Multi-View 2D Images

Temporal Resolution Enhancement of COMS Satellite Using Geo-Kompsat-2A Satellite Through Data-to-Data Translation

Assessing the Potential of Multi-Temporal Conditional Generative Adversarial Networks in SAR-to-Optical Image Translation for Early-Stage Crop Monitoring

Contact Info

Product

Resources

About