Performance Evaluation of Feature Detectors and Descriptors Beyond the Visible

Abstract

Feature detection and description algorithms represent an important milestone in most computer vision applications. They have been examined from various perspectives during the last decade. However, most studies focused on their performance when used on visible band imagery. This modality suffers considerably in poor lighting conditions and notably during night-time. Infrared cameras, which noticed a considerable proliferation in recent years, offer a viable alternative in such conditions. Understanding how the building blocks of computer vision applications behave in this modality would help the community accommodating them. For this reason, we carried out a performance analysis of the most commonly used feature detectors and descriptors beyond the visible. A dataset accounting for the various challenges on these algorithms has been generated. In addition, challenges inherent to the thermal modality have been considered, notably the non-uniformity noise. A comprehensive quantitative investigation into the performance of feature detectors and descriptors is therefore presented. This study would serve to filling the gap in the literature as most analyzes have been based on visible band imagery.

An Evaluation of Image Feature Detectors Based on Spatial Density and Temporal Robustness in Microsurgical Image Processing

“…This paper is focused on the evaluation of the spatial and temporal feature behavior. Unlike most state-of-the-art investigations [1], this paper is focused specifically in neurovascular videos, to reveal the best performing feature and which therefore can be recommended for usage in image feature-based applications on neurovascular data. The spatial evaluation aims to find the feature detector with the highest number of features detected in a region of interest (ROI).…”

Keyframe‐based thermal–inertial odometry

“…The proposed descriptor showed improved matching performance on thermal images, however, it was not able to cope with instances containing large motions. As detailed in Mouats, Aouf, Nam, and Vidas (), the application of feature methods, primarily designed for visual images, generally show lower matching performance when operating on rescaled thermal images. This lower matching performance can be partially attributed to the image rescaling operation which can either suppress weak image gradients, reducing the overall gradient variety present in the thermal frame, or can significantly change the appearance of successive images to make them inconsistent and cause tracking loss.…”

“…Utilization of a sparse set of points is more akin to feature‐based methods, where a set of points is carefully selected to better utilize the underlying structure in the scene and to maximize the tracking performance of points between frames. However, given the unconventional nature of full radiometric data and the reduced performance of feature‐based methods on rescaled thermal images (Mouats et al, ), the sparse set of points in the proposed approach are selected by utilizing the gradient information of the full radiometric data. Initially, image gradients are calculated over the whole image and then the image is divided into 32 × 32 pixel blocks.…”

“…In contrast, OKVIS is more suited to work through periods of data interruption (application of FFC) as it estimates the robot pose by optimizing over a horizon of temporally and spatially separated keyframes. However, to establish image correspondences, OKVIS relies on the BRISK feature detector and descriptor (Leutenegger, Chli, & Siegwart, ), which can perform poorly given the low contrast nature of the thermal data as shown in Mouats et al (), and may lead to tracking loss as depicted in Figure . Finally, DSO is an optimization‐based visual odometry approach that utilizes image gradients to identify and establish image correspondences.…”

“…Typically, the accumulated drift is corrected by finding loop closures using visual feature descriptors (Mur‐Artal et al, ). However, given the lower performance of visual feature detectors and descriptors on thermal camera data (Mouats et al, ), this might not be a viable approach for the correction of drift in the proposed thermal–inertial odometry estimation approach and, as an alternative, methods that employ map reuse using direct image data might be more applicable, as shown in Zubizarreta, Aguinaga, and Montiel (). Similarly, being monocular in nature, the proposed approach relies on the triangulation of points to estimate point depths, which can be challenging if the only points that can be tracked lie far away from the camera, hence, requiring sufficient motion between camera frames to occur to establish sufficient baselines.…”