Laplacian pyramid adversarial network for face completion

Wang, Qiang; Fan, Huijie; Sun, Gan; Yang, Cong; Tang, Yandong

doi:10.1016/j.patcog.2018.11.020

Cited by 50 publications

(17 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we demonstrate the performance and generated results of our model through experiments. Due to the work of this paper is mainly focused on generator, in order to make comparative experiments more meaningful, we mainly use different generators (CNN [6], Dilated CNN [19], Partial CNN [20], Hybrid dilated CNN [21] and Laplacian pyramid CNN [22] ) to compare with our model.…”

Section: Methodsmentioning

confidence: 99%

“…For example, the convolutional neural network (CNN) [6], dilated convolution [19] and partial convolution [20] enhanced the performance of the completion network by improving the performance of encoder responsible for feature extraction in the generator. Furthermore, in order to generate more detailed textures, hybrid dilated convolution (HDC) [21] and laplacian-pyramid-based convolution (LPC) [22] used U-Net based method and multi-scale theory to extract texture features in facial images. However, in the existing methods, the edges of the completion region regularly have structural and semantic discontinuities, since the spatial dependence of images is not used to extract features.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Facial Image Completion Using Bi-Directional Pixel LSTM

Xiang

Zhang

2020

IEEE Access

View full text Add to dashboard Cite

Structural features of facial images directly affect the performance of the image completion model. However, most existing work does not make full use of spatial dependence to extract features, and cause the semantics and structure of completion being inconsistent with the context. This paper addresses this issue using a bi-directional pixel long-short time memory (LSTM) network. Specifically, it consists of two LSTM subnetworks and can simultaneously scan the input image row by row or column by column, thereby the extracted features contain the dependence information among rows or among columns. Through a fusion operation of these features, the complete spatial dependence information is included. In addition, the parameters of the decoder and discriminator are automatically adjusted to accommodate the proposed bi-directional pixel LSTM. Finally, compared with existing state-of-the-art methods, experimental results show our superior performance.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Facial Image Completion Using Bi-Directional Pixel LSTM

Xiang

Zhang

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Currently, multi-scale-based methods have shown a significant development of applications in image inpainting. For instance, Wang et al [ 25 ] introduced a Laplacian-pyramid model to progressively restore images with different resolutions. Mo et al [ 26 ] introduced several multi-scale discriminators to generate the results containing more multi-scale information.…”

Section: Related Workmentioning

confidence: 99%

Non-Local and Multi-Scale Mechanisms for Image Inpainting

He¹,

Yin²

2021

Sensors

View full text Add to dashboard Cite

Recently, deep learning-based techniques have shown great power in image inpainting especially dealing with squared holes. However, they fail to generate plausible results inside the missing regions for irregular and large holes as there is a lack of understanding between missing regions and existing counterparts. To overcome this limitation, we combine two non-local mechanisms including a contextual attention module (CAM) and an implicit diversified Markov random fields (ID-MRF) loss with a multi-scale architecture which uses several dense fusion blocks (DFB) based on the dense combination of dilated convolution to guide the generative network to restore discontinuous and continuous large masked areas. To prevent color discrepancies and grid-like artifacts, we apply the ID-MRF loss to improve the visual appearance by comparing similarities of long-distance feature patches. To further capture the long-term relationship of different regions in large missing regions, we introduce the CAM. Although CAM has the ability to create plausible results via reconstructing refined features, it depends on initial predicted results. Hence, we employ the DFB to obtain larger and more effective receptive fields, which benefits to predict more precise and fine-grained information for CAM. Extensive experiments on two widely-used datasets demonstrate that our proposed framework significantly outperforms the state-of-the-art approaches both in quantity and quality.

show abstract

“…Yang et al [35] presented a block-wise procedural training scheme to address the difficulty of training a very deep generative model and adversarial loss annealing to improve inpainting result. Wang et al [33], designed a Laplacian-pyramid-based convolutional network framework to predict missing regions under different resolutions and adopted modified residual learning model to matching color, which works well on facial image inpainting. The above methods continuously improve the performance of inpainting result, however, they cannot address the artifacts including the lack of detail and blurry or distorted results.…”

Section: B Deep Learningmentioning

confidence: 99%

Facial Image Inpainting With Deep Generative Model and Patch Search Using Region Weight

Wei

Liu

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Facial image inpainting is a challenging task because the missing region needs to be filled by the new pixels with semantic information (e.g., noses and mouths). The traditional methods that involve searching for similar patches are mature but it is not suitable for semantic inpainting. Recently, the deep generative model-based methods have been able to implement semantic image inpainting although inpainting results are blurry or distorted. In this paper, through analyzing the advantages and disadvantages of the two methods, we propose a novel and efficient method that combines these two methods by a series connection, which searches for the most reasonable similar patch using the coarse image generated by the deep generative model. When training model, adding Laplace loss to standard loss accelerates model convergence. In addition, we define region weight (RW) when searching for similar patches, which makes edge connection more natural. Our method addresses the problem of blurred results in the deep generative model and dissatisfactory semantic information in the traditional methods. Our experiments, which used the CelebA dataset, demonstrate that our method can achieve realistic and natural facial inpainting results.INDEX TERMS Facial image inpainting, deep generative model, similar patch, region weight. I. INTRODUCTION

show abstract

Laplacian pyramid adversarial network for face completion

Cited by 50 publications

References 36 publications

Facial Image Completion Using Bi-Directional Pixel LSTM

Facial Image Completion Using Bi-Directional Pixel LSTM

Non-Local and Multi-Scale Mechanisms for Image Inpainting

Facial Image Inpainting With Deep Generative Model and Patch Search Using Region Weight

Contact Info

Product

Resources

About