Introduction of Deep Learning in Thermographic Monitoring of Cultural Heritage and Improvement by Automatic Thermogram Pre-Processing Algorithms

Abstract

The monitoring of heritage objects is necessary due to their continuous deterioration over time. Therefore, the joint use of the most up-to-date inspection techniques with the most innovative data processing algorithms plays an important role to apply the required prevention and conservation tasks in each case study. InfraRed Thermography (IRT) is one of the most used Non-Destructive Testing (NDT) techniques in the cultural heritage field due to its advantages in the analysis of delicate objects (i.e., undisturbed, non-contact and fast inspection of large surfaces) and its continuous evolution in both the acquisition and the processing of the data acquired. Despite the good qualitative and quantitative results obtained so far, the lack of automation in the IRT data interpretation predominates, with few automatic analyses that are limited to specific conditions and the technology of the thermographic camera. Deep Learning (DL) is a data processor with a versatile solution for highly automated analysis. Then, this paper introduces the latest state-of-the-art DL model for instance segmentation, Mask Region-Convolution Neural Network (Mask R-CNN), for the automatic detection and segmentation of the position and area of different surface and subsurface defects, respectively, in two different artistic objects belonging to the same family: Marquetry. For that, active IRT experiments are applied to each marquetry. The thermal image sequences acquired are used as input dataset in the Mask R-CNN learning process. Previously, two automatic thermal image pre-processing algorithms based on thermal fundamentals are applied to the acquired data in order to improve the contrast between defective and sound areas. Good detection and segmentation results are obtained regarding state-of-the-art IRT data processing algorithms, which experience difficulty in identifying the deepest defects in the tests. In addition, the performance of the Mask R-CNN is improved by the prior application of the proposed pre-processing algorithms.

Subdivision of Artistic Objectives of Oil Painting Creation under the Evaluation of Deep Learning Model

“…(2) At present, there are only a few studies on oil painting, but the main ones are as follows: [3] extracted the overall and local features and proposed the entropy balance (fusion) algorithm to classify the authors of Chinese paintings. [4] studied the different depth information features of paintings at different scales and frequency bands in the wavelet domain in order to classify paintings. [5] designed a related algorithm to classify the paintings of Shen Zhou, Tang Yin, Zhang Daqian, and other Chinese painting artists.…”

Blockchain Smart Contract to Prevent Forgery of Degree Certificates: Artificial Intelligence Consensus Algorithm

“…In addition, the difference between this and existing studies is that hybrid Blockc can be verified in all aspects. All transactions arising from hybrid Blockchain ca conducted privately, and if necessary, the transaction details can be opened verification [50]. Each transaction can be written only once because it uses a Blockc The role of cloud preprocessing is to support CNN Intelligent Agent for the purpose of the proposed research [47].…”

Thermographic Imaging in Cultural Heritage: A Short Review

“…For defect detection in the ancient marquetry sample, ESPCT outperforms PCT and SPCT. Another investigation of both types of marquetry samples was presented in [ 58 ]. In this work, pre-processing algorithms were conducted to compensate for the non-uniform backgrounds contained in the thermograms and highlight the most significant thermal footprints; then, a deep learning model named mask region-convolution neural network (Mask R-CNN) was trained for defect detection and segmentation.…”

“… Segmentation results of pulsed thermograms of marquetry samples using Mask R-CNN (Reprinted with permission from Ref. [ 58 ]. Copyright: 2021 Garrido, I.; Erazo-Aux, J.; Lagüela, S.; Sfarra, S.; Ibarra-Castanedo, C.; Pivarčiová, E.; Gargiulo, G.; Maldague, X.; Arias, P.).…”