Facial image inpainting is a challenging task because the missing region needs to be filled by the new pixels with semantic information (e.g., noses and mouths). The traditional methods that involve searching for similar patches are mature but it is not suitable for semantic inpainting. Recently, the deep generative model-based methods have been able to implement semantic image inpainting although inpainting results are blurry or distorted. In this paper, through analyzing the advantages and disadvantages of the two methods, we propose a novel and efficient method that combines these two methods by a series connection, which searches for the most reasonable similar patch using the coarse image generated by the deep generative model. When training model, adding Laplace loss to standard loss accelerates model convergence. In addition, we define region weight (RW) when searching for similar patches, which makes edge connection more natural. Our method addresses the problem of blurred results in the deep generative model and dissatisfactory semantic information in the traditional methods. Our experiments, which used the CelebA dataset, demonstrate that our method can achieve realistic and natural facial inpainting results.INDEX TERMS Facial image inpainting, deep generative model, similar patch, region weight. I. INTRODUCTION
Filling holes in an image is achieved in a manner similar to peeling the onion. The order of filling affects the image inpainting results, especially concerning the content of complex images. When highresolution images are used to extract edge information, they are susceptible to high-frequency information, such as complex textures and noise. Furthermore, edge information is extracted in different resolutions, while the main contour information of the image can be obtained more easily. In this paper, multi-resolution information is used to prioritize which target patches in an image to fill, which helps to elucidate the optimal sequence for image repair. Multi-resolution images provide more information than single-resolution images, and similar patches are computed on multi-resolution images to obtain multiple candidate patches. Similar patch calculations use a variety of information on colors, gradients, and boundaries to more accurately search for similar patches. We chose the most reasonable candidate patch by means of the structural similarity index measure (SSIM). When pasting the patch to fill the target region, we used graph cut technology to eliminate blockiness. Compared with the state-of-the-art repair algorithm, the experimental results prove that the proposed repair algorithm can repair the image very well.INDEX TERMS Exemplar-based inpainting technique, priority calculation, patch matching, graph cut, multi-resolution information.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.