Image‐to‐Geometry Registration: a Mutual Information Method exploiting Illumination‐related Geometric Properties

Abstract

This work concerns a novel study in the field of image-to-geometry registration. Our approach takes inspiration from medical imaging, in particular from multi-modal image registration. Most of the algorithms developed in this domain, where the images to register come from different sensors (CT, X-ray, PET), are based on Mutual Information, a statistical measure of non-linear correlation between two data sources. The main idea is to use mutual information as a similarity measure between the image to be registered and renderings of the model geometry, in order to drive the registration in an iterative optimization framework. We demonstrate that some illuminationrelated geometric properties, such as surface normals, ambient occlusion and reflection directions can be used for this purpose. After a comprehensive analysis of such properties we propose a way to combine these sources of information in order to improve the performance of our automatic registration algorithm. The proposed approach can robustly cover a wide range of real cases and can be easily extended.

A semantic-based platform for the digital analysis of architectural heritage

“…We plan to automatize some of these (especially the semantic annotation and the image registration) making use of the image-geometry mutual relation [33]. Readers can find a lot of resources about our platform in the following web page: http://www.map.archi.fr/nubes…”

Integration of 3D and multispectral data for cultural heritage applications: Survey and perspectives

“…The camera parameters are iteratively optimized and a new rendering is created until the registration is achieved. The precision of the ensuing registration is of the order of a few pixels, though the success of such methods greatly depends on the rendering strategy [116]. This method provides good results for visualization purposes, even using low quality images [114].…”

“…It is used to compare the 2D data to be mapped with a rendering of the 3D model. Many renderings have been used (depth map [114], gradient map [115], silhouette map, re ection map and other illumination-based renderings [116]). The camera parameters are iteratively optimized and a new rendering is created until the registration is achieved.…”

Automatic Registration of Images to Untextured Geometry Using Average Shading Gradients

“…Existing methods, in contrast, typically aim to maximize the statistical dependency between the photograph and a rendering, (e.g., Corsini et al 2009). The resulting registration criterion is dense, but leads to a highly non-convex optimization problem with many local optima, necessitating good initialization.…”

“…Registering images to 3D models of real-world objects or places is an important prerequisite for transferring information between images and a 3D model of the scene (Corsini et al 2009;Neugebauer and Klein 1999). For example, color information from images can be used to texture a 3D model that was previously acquired using range scans.…”