Toward a Data Fusion Index for the Assessment and Enhancement of 3D Multimodal Reconstruction of Built Cultural Heritage

Abstract

In the field of digital cultural heritage (DCH), 2D/3D digitization strategies are becoming more and more complex. The emerging trend of multimodal imaging (i.e., data acquisition campaigns aiming to put in cooperation multi-sensor, multi-scale, multi-band and/or multi-epochs concurrently) implies several challenges in term of data provenance, data fusion and data analysis. Making the assumption that the current usability of multi-source 3D models could be more meaningful than millions of aggregated points, this work explores a “reduce to understand” approach to increase the interpretative value of multimodal point clouds. Starting from several years of accumulated digitizations on a single use-case, we define a method based on density estimation to compute a Multimodal Enhancement Fusion Index (MEFI) revealing the intricate modality layers behind the 3D coordinates. Seamlessly stored into point cloud attributes, MEFI is able to be expressed as a heat-map if the underlying data are rather isolated and sparse or redundant and dense. Beyond the colour-coded quantitative features, a semantic layer is added to provide qualitative information from the data sources. Based on a versatile descriptive metadata schema (MEMoS), the 3D model resulting from the data fusion could therefore be semantically enriched by incorporating all the information concerning its digitization history. A customized 3D viewer is presented to explore this enhanced multimodal representation as a starting point for further 3D-based investigations.

Three-Dimensional Analysis for the Documentation of the Restoration of an Earthquake-Damaged Triptych

“…Investigation before restoration often requires both a comprehensive and a detailed analysis of the artwork, thus necessitating a multi-modal and multi-resolution approach to assess all issues. Moreover, data are often collected with different sensors at different times in variable contexts [46]. Therefore, understanding the quality of 3D results is essential to know what information can be extracted from 3D models obtained from different instruments.…”

An Open-Source Web Platform for 3D Documentation and Storytelling of Hidden Cultural Heritage

“…Thanks to its native functionalities that ranges from the visualisation of classified point clouds to the execution of simple geometric measurements, Potree is widely used in data sharing and visualization of digitised CH sites for management [27,39] or restoration planning purposes [40,41], enhancing its potential in team cooperation. This approach is particularly useful also for sharing information on sites located in remote areas or affected by limited accessibility [13,24,30,42] as well as for visualising the results of multisource 3D data integration [43,44]. To the best of the authors' knowledge, the literature has documented the widespread adoption of Potree as a simple 3D web viewer with simplified geometric tools [29,37,45], but its potential for storytelling and promotion to nonexpert users combining geospatial data with nongeomatics-derived information needs to be explored.…”

A Performance Comparison of 3D Survey Instruments for Their Application in the Cultural Heritage Field

“…Their applicability to different case studies depends on their performance in terms of resolution, gauge volume/area, acquisition time and the instrumentation cost. Given that the restoration of a work of art can last from a few months to several years, it is often necessary to repeat measurements with different instruments at different times in variable contexts (e.g., to document the restoration intervention, a shape survey is carried out before starting the work and after the work is finished) [ 12 ]. Moreover, the investigation of artworks usually requires both overall and detailed analyses of the surface, which then requires a multi-modal and multi-resolution approach to assess all issues.…”