Image inpainting algorithm based on image decomposition model and patch global structural information

Abstract

With the improvement of image processing tools and the flexibility of digital image editing, automatic image inpainting has found important applications in computer vision and has also become an important and challenging topic of research in image processing. Through the analysis of the exemplar-based image inpainting method, we found the calculation of priority of image patches to fill is beneficial to the textures than the geometric structure, while the geometric structure is more important to the filling than the textures, resulting in the problem of texture extension or the discontinuity of the geometric structure in the image. The image decomposition model is applied to decompose the image to inpaint into two components: texture and cartoon, which are then filled respectively. We improve the calculation of the priority of the texture component and the cartoon component, which make it suitable to the image character. In order to solve the mismatch problem, we propose a measurement based on global structure information of the target patch and the candidate patch in this paper. The mean and variance of the pixels in the image patch representing the global structure of image patches added to the measurement formula can improve the accuracy of the best matching patch to fill the missing region. We compared our model with the Criminisi algorithm and its improved one, through applying the algorithm to the synthetic images and the natural image. We also compute the PNSF and SSIM values between the original images and the inpainted images to verify the effectiveness of the proposed algorithm. The experimental results show that the proposed algorithm can get more perceptually aware inpainted results than the Criminisi algorithm and its improved algorithm.