Towards automatic classification of diffuse reflectance image cubes from paintings collected with hyperspectral cameras

Kleynhans, Tania; Messinger, David W.; Delaney, John K.

doi:10.1016/j.microc.2020.104934

Cited by 32 publications

(18 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The manuscript chosen for this work was the Laudario of Sant'Agnese (c. 1340), one of only three surviving illuminated books of this type (a laudario is a collection of hymns of praise), and which has individual illuminations (described as paintings throughout this paper for clarity) by at least two artists, which are now dispersed in several collections around the world [40][41][42]. The paintings used to build the training set (Additional file 1: Figure S1) include: [25,33].…”

Section: Data and Experimental Setupmentioning

confidence: 99%

“…The application of this workflow has been most successful when identification of the clusters which define potential classes is done manually, by an experienced user, in a reduced dimensional space with a subset of the spectra from the RIS data cube. Such processing, while successful, is also time-consuming [2,25]. Other automatic and more rapid algorithms to generate the class maps have shown promise but tend to only find about 70 to 80% of the classes in real paintings [25].…”

Section: Introductionmentioning

confidence: 99%

“…Such processing, while successful, is also time-consuming [2,25]. Other automatic and more rapid algorithms to generate the class maps have shown promise but tend to only find about 70 to 80% of the classes in real paintings [25]. All of these algorithms utilized in these workflows assume linear mixing and mixtures of pigments are treated as a single paint (a relatively consistent mixture of colored pigments) and hence a unique material.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An alternative approach to mapping pigments in paintings with hyperspectral reflectance image cubes using artificial intelligence

et al. 2020

Self Cite

View full text Add to dashboard Cite

Spectral imaging modalities, including reflectance and X-ray fluorescence, play an important role in conservation science. In reflectance hyperspectral imaging, the data are classified into areas having similar spectra and turned into labeled pigment maps using spectral features and fusing with other information. Direct classification and labeling remain challenging because many paints are intimate pigment mixtures that require a non-linear unmixing model for a robust solution. Neural networks have been successful in modeling non-linear mixtures in remote sensing with large training datasets. For paintings, however, existing spectral databases are small and do not encompass the diversity encountered. Given that painting practices are relatively consistent within schools of artistic practices, we tested the suitability of using reflectance spectra from a subgroup of well-characterized paintings to build a large database to train a one-dimensional (spectral) convolutional neural network. The labeled pigment maps produced were found to be robust within similar styles of paintings.

show abstract

Section: Data and Experimental Setupmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An alternative approach to mapping pigments in paintings with hyperspectral reflectance image cubes using artificial intelligence

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…The application of this workflow has been most successful when identification of the clusters which define potential classes is done manually, by an experienced user, in a reduced dimensional space with a subset of the spectra from the RIS data cube. Such processing, while successful, is also time-consuming [25,2]. Other automatic and more rapid algorithms to generate the class maps have shown promise but tend to only find about 70 to 80% of the classes in real paintings [25].…”

Section: Introductionmentioning

confidence: 99%

“…Such processing, while successful, is also time-consuming [25,2]. Other automatic and more rapid algorithms to generate the class maps have shown promise but tend to only find about 70 to 80% of the classes in real paintings [25]. All of these algorithms utilized in these workflows assume linear mixing and mixtures of pigments are treated as a single paint (a relatively consistent mixture of colored pigments) and hence a unique material.…”

Section: Introductionmentioning

confidence: 99%

An Alternative Approach to Mapping Pigments in Paintings With Hyperspectral Reflectance Image Cubes Using Artificial Intelligence

Kleynhans

Patterson

Dooley

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Abstract Spectral imaging modalities, including reflectance and X-ray fluorescence, play an important role in conservation science. In reflectance hyperspectral imaging, the data are classified into areas having similar spectra and turned into labeled pigment maps using spectral features and fusing with other information. Direct classification and labeling remain challenging because many paints are intimate pigment mixtures that require a non-linear unmixing model for a robust solution. Neural networks have been successful in modeling non-linear mixtures in remote sensing with large training datasets. For paintings, however, existing spectral databases are small and do not encompass the diversity encountered. Given that painting practices are relatively consistent within schools of artistic practices, we tested the suitability of using reflectance spectra from a subgroup of well-characterized paintings to build a large database to train a one-dimensional (spectral) convolutional neural network. The labeled pigment maps produced were found to be robust within similar styles of paintings.

show abstract

Comparison of macro x‐ray fluorescence and reflectance imaging spectroscopy for the semi‐quantitative analysis of pigments in easel paintings: A study on lead white and blue verditer

et al. 2023

View full text Add to dashboard Cite

Macroscopic x‐ray fluorescence imaging spectroscopy (MA‐XRF) and reflectance imaging spectroscopy (RIS) are important tools in the analysis of cultural heritage objects, both for conservation and art historical research purposes. The elemental and molecular distributions provided by MA‐XRF and RIS respectively, are particularly useful for the identification and mapping of pigments in easel paintings. While MA‐XRF has relatively established data processing methods based on modeling of the underlying physics, RIS data cannot be modeled with sufficient precision and its processing has considerable room for improvements. This work seeks to improve RIS data processing workflows in the short wavelength infrared range (SWIR, 1000–2500 nm) with a novel method that fits Gaussian profiles to pigment‐specific absorption features, and we compare its performance to MA‐XRF for the task of semi‐quantitative pigment mapping, evaluating their limits of detection (LODs) and the matrix effects that affect their signals. Two pigments are considered in this work, lead white and blue verditer, which are mapped in SWIR RIS using the first overtone of OH stretching of their primary compounds, hydrocerussite (Pb3(CO3)2(OH)2) and azurite (Cu3(CO3)2(OH)2), at 1447 and 1497 nm respectively, and in MA‐XRF using the Pb‐L and Cu‐K fluorescence signals. The methods are evaluated using two sets of custom‐prepared paint samples, as well as a 16th‐century painting, discussing the identification, mapping, and semi‐quantitative analysis of the considered pigments. We found SWIR RIS to be a pigment‐specific method with a longer linear range but inferior LODs and penetration depth when compared to MA‐XRF, the latter is often not capable of discriminating between different pigments with identical elemental markers. We furthermore present a novel color scale that allows the simultaneous visualization of signals above and below a confidence limit.

show abstract

Towards automatic classification of diffuse reflectance image cubes from paintings collected with hyperspectral cameras

Cited by 32 publications

References 18 publications

An alternative approach to mapping pigments in paintings with hyperspectral reflectance image cubes using artificial intelligence

An alternative approach to mapping pigments in paintings with hyperspectral reflectance image cubes using artificial intelligence

An Alternative Approach to Mapping Pigments in Paintings With Hyperspectral Reflectance Image Cubes Using Artificial Intelligence

Comparison of macro x‐ray fluorescence and reflectance imaging spectroscopy for the semi‐quantitative analysis of pigments in easel paintings: A study on lead white and blue verditer

Contact Info

Product

Resources

About