DeepGIN: Deep Generative Inpainting Network for Extreme Image Inpainting

Li, Chu Tak; Siu, Wan Chi; Liu, Zhi Song; Wang, Li Wen; Lun, Daniel Pak-Kong

doi:10.1007/978-3-030-66823-5_1

Cited by 16 publications

(6 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the inputs 𝐼 '+,' and 𝐼 % are both sparse. To extract a meaningful feature map, similar to [29], we used three spatial pyramid dilation blocks (SPD) after six convolution layers. Notably, SPD blocks contain parallel convolution layers with various dilation rates to extract a large receptive field from the given input image.…”

Section: ) Coarse Generatormentioning

confidence: 99%

E2F-GAN: Eyes-to-Face Inpainting via Edge-Aware Coarse-to-Fine GANs

et al. 2022

View full text Add to dashboard Cite

Face inpainting is a challenging task aiming to fill the damaged or masked regions in face images with plausibly synthesized contents. Based on the given information, the reconstructed regions should look realistic and more importantly preserve the demographic and biometric properties of the individual. The aim of this paper is to reconstruct the face based on the periocular region (eyes-to-face). To do this, we proposed a novel GAN-based deep learning model called Eyes-to-Face GAN (E2F-GAN) which includes two main modules: a coarse module and a refinement module. The coarse module along with an edge predictor module attempts to extract all required features from a periocular region and to generate a coarse output which will be refined by a refinement module. Additionally, a dataset of eyes-to-face synthesis has been generated based on the public face dataset called CelebA-HQ for training and testing. Thus, we perform both qualitative and quantitative evaluations on the generated dataset. Experimental results demonstrate that our method outperforms previous learning-based face inpainting methods and generates realistic and semantically plausible images. We also provide the implementation of the proposed approach to support reproducible research via (https://github.com/amiretefaghi/E2F-GAN).

show abstract

Section: ) Coarse Generatormentioning

confidence: 99%

E2F-GAN: Eyes-to-Face Inpainting via Edge-Aware Coarse-to-Fine GANs

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Figure 4 presents the overall process of constructing the GIN for restoring s atellite sea surface temperature data. For the robust and stable restoration of missing regions, the GIN model adopts the state-of-the-art two-stage coarse-to-fine network architecture [17,18]. It consists of a coarse network that makes an initial coarse reconstruction and a refinement network that takes the coarse reconstruction as input and then predicts the refined results with details and textures.…”

Section: Generative Inpainting Network (Gin)mentioning

confidence: 99%

“…Second, to train our GIN for restoring the input SST images as realistically as possible, we defined and effectively combined multiple loss functions; in particular, we developed a novel "sea surface loss" that enables the application of the information collected from the surrounding locations of the ocean surface regions (no land region) to the restoration of missing regions (Section 2.3). Third, our GIN is composed of two-stage networks [17,18], so-called coarse network and refinement network, to obtain a realistic and coherent completed SST image; such twostage network architecture has proven effective for restoring missing regions with highly complex and arbitrary shapes. Our proposed GIN employs a multistage transfer learning framework, which consists of pretraining and fine-tuning stages.…”

Section: Introductionmentioning

confidence: 99%

Restoration of Missing Patterns on Satellite Infrared Sea Surface Temperature Images Due to Cloud Coverage Using Deep Generative Inpainting Network

Kang

Choi²,

Choi

2021

JMSE

View full text Add to dashboard Cite

In this paper, we propose a novel deep generative inpainting network (GIN) trained under the framework of generative adversarial learning, which is optimized for the restoration of cloud-disturbed satellite sea surface temperature (SST) imagery. The proposed GIN architecture can achieve accurate and fast restoration results. The proposed GIN consists of rough and fine reconstruction stages to promote the details and textures of missing (clouded) regions in SST images. We also propose a nov el preprocessing strategy that replaces the land areas with the average value of daily oceanic surface temperatures for improving restoration accuracy. To learn the proposed GIN, we developed a novel approach that combines multiple loss functions well suited for improving the restoration quality over missing SST information. Our results show that the difference in temperature between restored and actual satellite image data was no larger than 0.7 °C in monthly average values, which suggests excellent resilience against the missing sea surface temperature data. The proposed GIN has a faster restoration time and is feasible for real-time ocean-related applications. Furthermore, the computational cost of restoring SST images is much lower than the popular interpolation methods.

show abstract

“…The proposed model [17] consists of two generators and two discriminators. The overall proposed architecture is displayed in…”

Section: Deepinpaintingt1mentioning

confidence: 99%

AIM 2020 Challenge on Image Extreme Inpainting

Ntavelis¹,

Romero²,

Bigdeli³

et al. 2020

Preprint

View full text Add to dashboard Cite

This paper reviews the AIM 2020 challenge on extreme image inpainting. This report focuses on proposed solutions and results for two different tracks on extreme image inpainting: classical image inpainting and semantically guided image inpainting. The goal of track 1 is to inpaint large part of the image with no supervision. Similarly, the goal of track 2 is to inpaint the image by having access to the entire semantic segmentation map of the input. The challenge had 88 and 74 participants, respectively. 11 and 6 teams competed in the final phase of the challenge, respectively. This report gauges current solutions and set a benchmark for future extreme image inpainting methods.

show abstract

DeepGIN: Deep Generative Inpainting Network for Extreme Image Inpainting

Cited by 16 publications

References 30 publications

E2F-GAN: Eyes-to-Face Inpainting via Edge-Aware Coarse-to-Fine GANs

E2F-GAN: Eyes-to-Face Inpainting via Edge-Aware Coarse-to-Fine GANs

Restoration of Missing Patterns on Satellite Infrared Sea Surface Temperature Images Due to Cloud Coverage Using Deep Generative Inpainting Network

AIM 2020 Challenge on Image Extreme Inpainting

Contact Info

Product

Resources

About