Free-Form Image Inpainting with Gated Convolution

Yu, Jiahui; Lin, Zhe; Yang, Jimei; Xiaohui, Shen; Lu, Xin; Thomas, Huang

doi:10.48550/arxiv.1806.03589

Cited by 79 publications

(196 citation statements)

References 0 publications

Supporting

Mentioning

196

Contrasting

Order By: Relevance

“…In recent years, Generative Adversarial Networks (GANs) [4] have made great success in image synthesis [28,27,21]. Isola et al [9] first introduced conditional GANs [17] to solve the image-to-image generation task, which was extended to high-resolution image synthesis [25].…”

Section: Image Synthesismentioning

confidence: 99%

See 1 more Smart Citation

PISE: Person Image Synthesis and Editing with Decoupled GAN

Zhang

Lai

et al. 2021

Preprint

View full text Add to dashboard Cite

Person image synthesis, e.g., pose transfer, is a challenging problem due to large variation and occlusion. Existing methods have difficulties predicting reasonable invisible regions and fail to decouple the shape and style of clothing, which limits their applications on person image editing. In this paper, we propose PISE, a novel two-stage generative model for Person Image Synthesis and Editing, which is able to generate realistic person images with desired poses, textures, or semantic layouts. For human pose transfer, we first synthesize a human parsing map aligned with the target pose to represent the shape of clothing by a parsing generator, and then generate the final image by an image generator. To decouple the shape and style of clothing, we propose joint global and local per-region encoding and normalization to predict the reasonable style of clothing for invisible regions. We also propose spatial-aware normalization to retain the spatial context relationship in the source image. The results of qualitative and quantitative experiments demonstrate the superiority of our model on human pose transfer. Besides, the results of texture transfer and region editing show that our model can be applied to person image editing. The code is available for research purposes at https://github.com/Zhangjinso/PISE.

show abstract

Section: Image Synthesismentioning

confidence: 99%

“…Gated convolution can learn a dynamic selection mechanism for each spatial location, which is suitable for unaligned generation tasks [28,27,29]. Finally, with the deformed parsing feature F d , the generated human parsing map S g is delivered by a decoder with standard configuration.…”

Section: Parsing Generatormentioning

confidence: 99%

PISE: Person Image Synthesis and Editing with Decoupled GAN

Zhang

Lai

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Generative adversarial network (GAN) [9] based on convolutional neural network (CNN) has led a series of breakthroughs for various applications including imageto-image translation [5,14,32,51] and image inpainting [35,38,45]. However, due to the instability problem during the training procedure, it is still a challenge to produce high-quality images [37].…”

Section: Introductionmentioning

confidence: 99%

Image Generation with Self Pixel-wise Normalization

Yeo¹,

Sagong²,

Park³

et al. 2022

Preprint

View full text Add to dashboard Cite

Region-adaptive normalization (RAN) methods have been widely used in the generative adversarial network (GAN)-based image-to-image translation technique. However, since these approaches need a mask image to infer the pixel-wise affine transformation parameters, they cannot be applied to the general image generation models having no paired mask images. To resolve this problem, this paper presents a novel normalization method, called self pixel-wise normalization (SPN), which effectively boosts the generative performance by performing the pixel-adaptive affine transformation without the mask image. In our method, the transforming parameters are derived from a self-latent mask that divides the feature map into the foreground and background regions. The visualization of the selflatent masks shows that SPN effectively captures a sin-

show abstract

“…Generative adversarial networks (GAN) [7] is an algorithmic framework that shows impressive performance in diverse applications including image-to-image translation [4,15,49], text-to-image translation [13,31], and image inpainting [33,36,45]. Generally, the GAN consists of two different networks, called generator and discriminator, which are trained with opposite goals.…”

Section: Introductionmentioning

confidence: 99%

Effective Shortcut Technique for GAN

Park,

Yoo,

Shin

2022

Preprint

View full text Add to dashboard Cite

In recent years, generative adversarial network (GAN)-based image generation techniques design their generators by stacking up multiple residual blocks. The residual block generally contains a shortcut, i.e. skip connection, which effectively supports information propagation in the network. In this paper, we propose a novel shortcut method, called the gated shortcut, which not only embraces the strength point of the residual block but also further boosts the GAN performance. More specifically, based on the gating mechanism, the proposed method leads the residual block to keep (or remove) information that is relevant (or irrelevant) to the image being generated. To demonstrate that the proposed method brings significant improvements in the GAN performance, this paper provides extensive experimental results on the various standard datasets such as CIFAR-10, CIFAR-100, LSUN, and tiny-ImageNet. Quantitative evaluations show that the gated shortcut achieves the impressive GAN performance in terms of Frechet inception distance (FID) and Inception score (IS). For instance, the proposed method improves the FID and IS scores on the tiny-ImageNet dataset from 35.13 to 27.90 and 20.23 to 23.42, respectively.

show abstract

Free-Form Image Inpainting with Gated Convolution

Cited by 79 publications

References 0 publications

PISE: Person Image Synthesis and Editing with Decoupled GAN

PISE: Person Image Synthesis and Editing with Decoupled GAN

Image Generation with Self Pixel-wise Normalization

Effective Shortcut Technique for GAN

Contact Info

Product

Resources

About