Evaluation of the effectiveness of HDR tone-mapping operators for photogrammetric applications

Abstract

Abstract:The ability of High Dynamic Range (HDR) imaging to capture the full range of lighting in a scene has prompted an increase in its use for Cultural Heritage (CH) applications. Photogrammetric techniques allow the semi-automatic production of three-dimensional (3D) models from a sequence of images. Current photogrammetric methods are not always effective in reconstructing images under harsh lighting conditions, as significant geometric details may not have been captured accurately within under-and over-exposed regions of the image. HDR imaging offers the possibility to overcome this limitation, however the HDR images need to be tone-mapped before they can be used within existing photogrammetric algorithms. In this paper we evaluate four different HDR tone-mapping operators (TMOs) that have been used to convert raw HDR images into a format suitable for state-of-the-art algorithms, and in particular keypoint detection techniques. The evaluation criteria used are the number of keypoints, the number of valid matches achieved and the repeatability rate. The comparison considers two local and two global TMOs. HDR data from four CH sites were used: Kaisariani Monastery (Greece), Asinou Church (Cyprus), Château des Baux (France) and Buonconsiglio Castle (Italy).Key words: high dynamic range (HDR) imaging, HDR tone-mapping, keypoint detection, image-based 3D reconstruction Resumen:Las posibilidades que ofrecen las imágenes de alto rango dinámico (HDR) para registrar la totalidad del rango de iluminación de una escena han propiciado su creciente uso en aplicaciones de patrimonio cultural. Los métodos fotogramétricos actuales permiten la producción semi-automática de modelos tridimensionales (3D) a partir de una secuencia de imágenes. Sin embargo, éstos presentan serias limitaciones en escenas con iluminación dura, resultando en consecuencia la aparición de zonas expuestas o sobreexpuestas. En este tipo de condiciones, el uso de imágenes HDR ofrece la posibilidad de superar este problema. Para evaluar su potencialidad, se presentan en este artículo cuatro operadores diferentes de mapeado tonal (tone-mapping) en imágenes HDR, conocidos como TMOs, cuya misión es convertir las imágenes HDR crudas en un formato adecuado para su uso en algoritmos de vanguardia, y en particular en técnicas de detección de entidades. Los criterios de evaluación que se utilizan para analizar su potencialidad son: el número de entidades detectadas, el número de correspondencias válidas y su índice de repetibilidad. En la comparación se incluyen TMOs, dos locales y dos globales. Se utilizan datos HDR tomados en cuatro sitios patrimoniales: el monasterio de Kaisariani (Grecia), la iglesia de Asinou (Chipre), el castillo de los Baux (Francia) y el castillo de Buonconsiglio (Italia).Palabras clave: toma de imágenes de alto rango dinámico (HDR), mapeado tonal HDR, detección de entidades, reconstrucción 3D basada en imágenes

Learning-based adaptive tone mapping for keypoint detection

“…Then, we compare our model with state-of-the-art perception based TMOs: ChiuTMO [15], DragoTMO [14], ReinhardTMO [6] and MantiukTMO [7]. We considered these TMOs as they have been previously applied for HDR evaluation studies [5,9] for similar keypoint detection task.…”

“…As a result, HDR linear values are not appropriate when used with LDR-optimized keypoint detection algorithms. In such a scenario, a plausible solution investigated by recent studies [2,3,5] is to convert HDR into an LDR representation using a Tone Mapping Operator (TMO) [1].…”

Fifty shades of HDR

“…Most [39] TMOs have been developed for humans to view HDR content [8], therefore when processed with computer vision algorithms the results might be different from what is expected. As Fig.…”

“…4 shows, such perceptually based TMOs might give worse results than the LDR image (eg Reinhard [36]) because parts of the scene which are perceptually important are highlighted, or the TMO is simply poor at reproducing detail in bright or dark areas of the scene [37]. Furthermore, when a TMO is finely-tuned towards enhancing small contrast or details (eg Fattal [38]), even sensor noise can be detected as a feature [39].…”

HDR video past, present and future: A perspective

“…Perceptual encoding can exploit knowledge of the HVS, such as the fact the HVS is able to discern luminance threshold differences more clearly in darker areas than brighter ones to achieve efficient encoding [57]. However, for computer vision applications, such perceptual encoding may result in detail in the scene being missed or over-accentuated by the computer vision algorithms [44,58,59]. Simple linear encoding is not the answer either.…”