Semantic Image Inpainting through Improved Wasserstein Generative Adversarial Networks

Abstract

Image inpainting is the task of filling-in missing regions of a damaged or incomplete image. In this work we tackle this problem not only by using the available visual data but also by incorporating image semantics through the use of generative models. Our contribution is twofold: First, we learn a data latent space by training an improved version of the Wasserstein generative adversarial network, for which we incorporate a new generator and discriminator architecture. Second, the learned semantic information is combined with a new optimization loss for inpainting whose minimization infers the missing content conditioned by the available data. It takes into account powerful contextual and perceptual content inherent in the image itself. The benefits include the ability to recover large regions by accumulating semantic information even it is not fully present in the damaged image. Experiments show that the presented method obtains qualitative and quantitative top-tier results in different experimental situations and also achieves accurate photo-realism comparable to state-of-the-art works.

On Using Perceptual Loss within the U-Net Architecture for the Semantic Inpainting of Textile Artefacts with Traditional Motifs

“…Similarity is defined based on a weighted context loss from the "good" pixel count and a prior loss for penalizing unrealistic output in accordance to the training images. The Wasserstein-1 metric for comparing the distributions of the generated versus the real image is often included in the GAN [8]. Boundary equilibrium GAN is believed to be an improved version of this architecture for semantic image inpainting [9].…”

Image Inpainting: A Review

“…On the RBG-D images, Dhamo et al [48] use CNN and GAN model to generate the background of a scene by removing the object in the foreground image as performed by many methodsod of motion detection using background subtraction [49][50][51] . In order to complete the missing regions in the image, Vitoria et al [52] proposed an improved version of the Wasserstein GAN with the incorporation of Discriminator and Generator architecture. In the same context, but on sea surface temperature (SST) images, the Dong et al [53] proposed a Deep Convolutional Generative adversarial network (DCGAN) for filing the missing parts of the images.…”

A method of inpainting moles and acne on the high‐resolution face photos

“…In the RBG-D image, Vitoria et al [27] proposed a modified version of the [7]. In the ocean temperature image, Dong et al [28] proposed a deep convolutional adversarial network to fill missing areas in the image.…”

“…In the RBG‐D image, Vitoria et al . [27] proposed a modified version of the [7]. In the ocean temperature image, Dong et al .…”