Auto-inpainting heritage scenes: a complete framework for detecting and infilling cracks in images and videos with quantitative assessment

Abstract

The need for preservation of cultural heritage has necessitated the research on digitally repairing the photographs of damaged monuments. In this paper, we first propose a technique for automatically detecting the cracked regions in photographs of monuments. Unlike the usual practice of manually selecting the mask for inpainting, the detected regions are supplied to an inpainting algorithm. Thus, the process of digitally repairing the cracked regions that physical objects have, using inpainting, is completely automated. The detection of cracked regions is based on comparison of patches, for which we use a measure derived from the edit distance, which is a popular string metric used in the area of text mining. Further, we extend this method to perform inpainting of video frames by making use of the scaleinvariant feature transform and homography. We consider the camera to move while capturing video of the heritage site, as such videos are typically captured by novices, hobbyists and tourists. Finally, we also propose a video quality measure to quantify the temporal consistency of the inpainted video. Experiments have been carried out on videos captured from the heritage site at Hampi, India.

Inpainting of Structural Reconstruction of Monuments Using Singular Value Decomposition Refinement of Patches

“…Further these patches can be refined [20] using some technique to retain the inherent features. Application of inpainting algorithm for [21] reconstruction of heritage sites is an unique idea. Most of the monuments at the verge of destruction in India are either from medieval or earlier.…”

Line-Drawing Enhanced Interactive Mural Restoration For Dunhuang Mogao Grottoes

“…Traditionally, it is done manually, which is subjective and enervating (Padalkar and Joshi, 2015) (Turakhia et al, 2012).…”