Neural reflectance transformation imaging

Abstract

Reflectance transformation imaging (RTI) is a computational photography technique widely used in the cultural heritage and material science domains to characterize relieved surfaces. It basically consists of capturing multiple images from a fixed viewpoint with varying lights. Handling the potentially huge amount of information stored in an RTI acquisition that consists typically of 50–100 RGB values per pixel, allowing data exchange, interactive visualization, and material analysis, is not easy. The solution used in practical applications consists of creating “relightable images” by approximating the pixel information with a function of the light direction, encoded with a small number of parameters. This encoding allows the estimation of images relighted from novel, arbitrary lights, with a quality that, however, is not always satisfactory. In this paper, we present NeuralRTI, a framework for pixel-based encoding and relighting of RTI data. Using a simple autoencoder architecture, we show that it is possible to obtain a highly compressed representation that better preserves the original information and provides increased quality of virtual images relighted from novel directions, especially in the case of challenging glossy materials. We also address the problem of validating the relight quality on different surfaces, proposing a specific benchmark, SynthRTI, including image collections synthetically created with physical-based rendering and featuring objects with different materials and geometric complexity. On this dataset and as well on a collection of real acquisitions performed on heterogeneous surfaces, we demonstrate the advantages of the proposed relightable image encoding.

The earliest beetle †Coleopsis archaica (Insecta: Coleo­ptera) – morphological re-evaluation using Reflectance Transformation Imaging (RTI) and phylogenetic assessment

“…This could facilitate a more widespread adoption in paleoentomology. Even though the technique is not new, interesting new features such as the radial basis function algorithm and improved methods to virtually relight surfaces ('neural RTI', Dulecha et al 2020) have been added recently to further improve the performance of RTI. This makes RTI a very useful tool to examine and document small-scale surface structures of moderately to heavily compressed insect fossils.…”

Reflectance Transformation Imaging as a Tool for Computer-Aided Visual Inspection

“…This last approach implements the natural modes of vibration to form the basis of decomposition. More recently, local interpolation approaches have shown interesting results, such as the Radial Basis Function [48] or even machine learning approaches [49].…”

Supervised segmentation of RTI appearance attributes for change detection on cultural heritage surfaces

“…Furthermore, these coefficients have been selected as more representative of cultural heritage applications. Even though there is evidence that there exist more accurate models [22,27,28], especially concerning specular surfaces, PTM and HSH provide a more simplified evaluation of the proposed methodology, are openly available and therefore address a larger audience. In other works, their performance has been assessed with the scope to compare them with new and more advanced models and on surfaces with high gloss and specularity; in later cases, HSH has proven to perform better than PTM [22,27,28].…”