An improved nonlinear model for image inpainting

Abstract

An important task in image processing is the process of filling in missing parts of damaged images based on the information obtained from the surrounding areas. It is called inpainting. The goals of inpainting are numerous such as removing scratches in old photographic image, removing text and logos, restoration of damaged paintings. In this paper we present a nonlinear diffusion model for image inpainting based on a nonlinear partial differential equation as first proposed by Perona and Malik in [8]. In our previous work [3] the existence, uniqueness and regularity of the solution for the proposed mathematical model are established in an Hilbert space. The discretization of the partial differential equation of the proposed model is performed using finite elements method and finite differences method. For finite differences method our model is very simple to implement, fast, and produces nearly identical results to more complex, and usually slower, known methods. However for finite elements method we observe that it requires large computational cost, especially for high-resolution images. To avoid this slowing problem, domain decomposition algorithm has been proposed, aiming to split one large problem into many smaller problems. To illustrate the effective performance of our method, we present some experimental results on several images.

Enhancing image inpainting through image decomposition and deep neural networks

“…This work extends our existing efforts for inpainting problems [4]. We establish a combination of the finite element method with machine learning techniques to solve PDE for inpainting problems.…”

“…However, the algorithm can be computationally intensive and requires good initialization of the known parts of the image to produce good results. It should be noted that there are several other mathematical models by nonlinear partial differential equations for image processing which have recently been derived from that of Perona and Malik and which have made it possible to improve their performance [4][5][6]. We will focus in the following on one of these models to build our powerful algorithm.…”

“…where the diffusion function ρ corresponds to the one used in the modified Perona-Malik model, more details can be found in [4]. This model has been successfully used in image processing to fill missing regions [4]; however, its main drawback is that it does not give a good result for images with larger sizes [9], we have coupled the DDM to the finite element method for this model and we obtained better results for high-resolution results for high resolution images. In this work, we introduce DNN within the DDM especially for the finite element method to ameliorate the accuracy and control the choice of the subdomains number.…”

Efficient Data Optimisation for Harmonic Inpainting with Finite Elements

“…This means that the required memory for our optimisation scales linearly in the number of pixels. We achieve this by applying a nested conjugate gradient solver to (5). The inner iterations are effectively solving inpainting problems due to the Laplace equation instead of inverting the matrix from the FEM formulation explicitly.…”

“…Related Work. Finite elements have been successfully used for PDE models for image denoising [3,18,27] and restoration [5,32]. However, to our knowledge they have not been applied to PDE-based image approximation from sparse data.…”