Photogrammetry Now and Then – From Hand-Crafted to Deep-Learning Tie Points –

Abstract

Abstract. Historical images provide a valuable source of information exploited by several kinds of applications, such as the monitoring of cities and territories, the reconstruction of destroyed buildings, and are increasingly being shared for cultural promotion projects through virtual reality or augmented reality applications. Finding reliable and accurate matches between historical and present images is a fundamental step for such tasks since they require to co-register the present 3D scene with the past one. Classical image matching solutions are sensitive to strong radiometric variations within the images, which are particularly relevant in these multi-temporal contexts due to different types of sensitive media (film/sensors) employed for the image acquisitions, different lighting conditions and viewpoint angles. In this work, we investigate the actual improvement provided by recent deep learning approaches to match historical and nowadays images. As learning-based methods have been trained to find reliable matches in challenging scenarios, including large viewpoint and illumination changes, they could overcome the limitations of classic hand-crafted methods such as SIFT and ORB. The most relevant approaches proposed by the research community in the last years are analyzed and compared using pairs of multi-temporal images.

A Digital 4D Information System on the World Scale: Research Challenges, Approaches, and Preliminary Results

“…These approaches usually fail to deal with less distinctive architecture, such as houses of a similar style, or fail when few images are available. Even using more advanced machine learning approaches or by combining different algorithms such as DELF and SuperGlue [130] only allows the realization of prototypic scenarios [6,131,132]. Another approach bypasses the modelling stage to generate visualizations directly from imagery [127,133,134], e.g., by transforming or assembling image content (recent image generators like DALL-E [135]).…”

Solving photogrammetric cold cases using AI-based image matching: New potential for monitoring the past with historical aerial images

“…Two different methods were considered to process historical aerial images and especially to robustly find tie points between sequential image pairs. Both approaches have already shown good results on historical terrestrial images (Maiwald, 2022;Morelli et al, 2022;Maiwald et al, 2023). The first method combines the tie point extractor SuperPoint (DeTone et al, 2018) with the feature matching method Su-perGlue (Sarlin et al, 2020) and will in the following be referred to as SuperGlue.…”

“…Again, training is done on the MegaDepth dataset and not modified during the experiments. Usually, the approach finds less correct matched features than SuperGlue but provides a higher absolute number of detected feature points (Maiwald et al, 2021;Morelli et al, 2022).…”

“…This is mainly due to the following persistent issues. Digitized historical (aerial) images often suffer from poor image quality resulting in close-to-zero correct feature matches (Maiwald, 2019;Zhang et al, 2021;Morelli et al, 2022). Further, even when applying learned feature matching methods, the texture quality and visual changes, particularly in forested areas, remain challenging.…”

Digitizing Historical Aerial Images: Evaluation of the Effects of Scanning Quality on Aerial Triangulation and Dense Image Matching

“…Although these methods proved to be effective for image matching tasks, in the last years, deep-learning approaches and hybrid processing pipelines have also emerged and been demonstrated to often overcome common limitations of traditional methods, e.g., multi-temporal data, radiometric changes, etc. [30][31][32][33][34]. Learning-based methods started to be applied to historical images for 4D urban reconstruction purposes [26,35,36].…”