An alternative approach to mapping pigments in paintings with hyperspectral reflectance image cubes using artificial intelligence

Abstract

Spectral imaging modalities, including reflectance and X-ray fluorescence, play an important role in conservation science. In reflectance hyperspectral imaging, the data are classified into areas having similar spectra and turned into labeled pigment maps using spectral features and fusing with other information. Direct classification and labeling remain challenging because many paints are intimate pigment mixtures that require a non-linear unmixing model for a robust solution. Neural networks have been successful in modeling non-linear mixtures in remote sensing with large training datasets. For paintings, however, existing spectral databases are small and do not encompass the diversity encountered. Given that painting practices are relatively consistent within schools of artistic practices, we tested the suitability of using reflectance spectra from a subgroup of well-characterized paintings to build a large database to train a one-dimensional (spectral) convolutional neural network. The labeled pigment maps produced were found to be robust within similar styles of paintings.

Low-Cost Multispectral System Design for Pigment Analysis in Works of Art

“…To validate the proposed system, the HSI reflectance image cube of the Pentecost , a 14th-Century illumination, was used. This painting has been studied in detail by conservators and conservation and imaging scientists at the National Gallery of Art, Washington DC, and the Getty Museum [ 10 ]. A complete map of the pigments present was derived by using HSI (visible-to-near-infrared (VNIR)), XRF imaging, point-fiber optics spectroscopy (350 to 2500 nm), and Raman spectroscopy.…”

“…The color image of the illumination is displayed in Figure 1 . The pigment database created in [ 10 ] for the development of a pigment identification neural network was used for the spectral band selection study described in Section 2.1 . The database consisted of 17,000 spectra representing 25 pigments and pigment mixtures (classes) that was selected and labeled from 4 well-characterized illuminations.…”

“…If the score of a newly selected band is similar to the previous score, the new band offers little additional contribution, indicating that the number of useful bands has been determined. The BPI method was applied to the pigment database [ 10 ], and the scores suggested that 10 to 15 bands would be necessary. However, these scores do not consider the spectral profile or bandwidth of the potential MSI filters.…”

“…The spectral bands for the MSI system were chosen after a comprehensive band selection study. The proposed system model was validated using HSI reflectance data collected in a previous study [ 10 ]. The painting, a 14th-Century illumination, was studied in detail with various imaging modalities, and the results here were compared to the HSI “ground truth” data.…”

Nondestructive Evaluation of Heritage Object Coatings with Four Hyperspectral Imaging Systems

“…Other methods such as partial least squares (PLS) regression or partial least squares discriminant analysis (PLSDA) establish relationships between the spectral data and the constituents of the sample [23]. Alternatively, hyperspectral images can be analysed using artificial intelligence as proposed by Kleynhans et al [24]. However, large training datasets are needed to successfully explore convolutional neural networks (CNNs).…”