Virtual restoration of paintings using adaptive adversarial neural network

Abstract

Over time, the visual quality of the paintings deteriorates. Cracks and loss of paint are the main types of damages that worsen the visual component of the painting. One of the ways to return the authentic appearance of paintings is a virtual restoration. Virtual restoration consists of two main stages: detecting deterioration and their removal. In this research, we investigate the possibility of applying deep learning-based methods for virtual restoration. To detect cracks we use a combination of convolutional (MCN) and autoencoder neural networks based on U-NET architecture, and to remove them, an adaptive adversarial network (aGAN). Also, in this work, we propose an original way of training an adversarial neural network, which allows us to apply it more successfully in practice. A series of experiments shows encouraging results compared to known methods and confirms the high efficiency of deep learning.