Robust Vision-Based Autonomous Navigation, Mapping and Landing for MAVs at Night

Daftry, Shreyansh; Das, Manash Pratim; Delaune, Jeff; Sorice, Cristina; Hewitt, Robert A.; Reddy, Shreetej; Lytle, Daniel; Gu, Elvin; Matthies, Larry

doi:10.1007/978-3-030-33950-0_21

Cited by 4 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For our application, we experimented with popular monocular VO/VIO algorithms (DSO [24], SVO2.0 [31], VINS-MONO [32], MSCKF [33]) to localize our MAV at night [11]. While these pipelines vary widely in their approach, there exists a common point of failure which can be contributed to the failure to track features from one frame to another under the three characteristic challenges as described above.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…growing interest in the computer vision community to use thermal cameras -ranging from object detection [4] and activity recognition [5] to stereo [6], visual odometry [7], [8], SLAM [9], [10] and 3D mapping [11]. See [12] for a comprehensive review on computer vision research and applications with thermal infrared cameras.…”

Section: Introductionmentioning

confidence: 99%

“…Despite the aforementioned advantages, the performance of traditional computer vision techniques developed for electro-optical imagery does not directly translate to the thermal domain [11], [14]. This is primarily because these cameras have several characteristics that are challenging for vision algorithms.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Online Photometric Calibration of Automatic Gain Thermal Infrared Cameras

Das,

Matthies,

Daftry

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Thermal infrared cameras are increasingly being used in various applications such as robot vision, industrial inspection and medical imaging, thanks to their improved resolution and portability. However, the performance of traditional computer vision techniques developed for electro-optical imagery does not directly translate to the thermal domain due to two major reasons: these algorithms require photometric assumptions to hold, and methods for photometric calibration of RGB cameras cannot be applied to thermal-infrared cameras due to difference in data acquisition and sensor phenomenology. In this paper, we take a step in this direction, and introduce a novel algorithm for online photometric calibration of thermalinfrared cameras. Our proposed method does not require any specific driver/hardware support and hence can be applied to any commercial off-the-shelf thermal IR camera. We present this in the context of visual odometry and SLAM algorithms, and demonstrate the efficacy of our proposed system through extensive experiments for both standard benchmark datasets, and real-world field tests with a thermal-infrared camera in natural outdoor environments.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Online Photometric Calibration of Automatic Gain Thermal Infrared Cameras

Das,

Matthies,

Daftry

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Over the course of the past several years, numerous applications of thermal cameras in machine vision have been developed [52]- [59]. These applications includes object recognition, human activity detection, visual odometry and Simultaneous Localization And Mapping (SLAM) [60][61][62] Thermal sensors are one type of imaging technology that, in certain conditions, can function even when there is no visible light.…”

Section: Thermal Camera Visionmentioning

confidence: 99%

Analysis of modern learning-based multimodal fusion algorithms for neuromorphic vision sensors

AlRemeithi¹,

Swamidoss²,

Mansoori³

et al. 2023

SPIE Future Sensing Technologies 2023

View full text Add to dashboard Cite

Image enhancement is an ongoing research problem that the community is addressing through the development of fusion algorithms. Such techniques typically involve the reconstruction of RGB images by removing environmental artifacts and enhancing desired features. Infrared imagery is also widely used to improve situational awareness in low visibility scenarios. Recently, learning-based approaches are used for fusion purposes to extract meaningful representations from images and capture latent features that could otherwise be inaccessible using conventional image processing algorithms. The inadequacies of RGB images in these algorithms’ pipelines are still obvious, despite the fact that the viability of RGB-Infrared image fusion has been thoroughly demonstrated in the literature. For example, RGB images often have artefacts like sudden changes in exposure or motion blur when the illumination changes or sudden changes in the scene. A novel imaging sensor operating in the visible light spectrum has been developed to address these issues. In this paper, we explore the cutting-edge paradigm of Neuromorphic Vision Sensors (NVS), a class of asynchronous analog imaging platforms that operate based on the change of pixel luminosity within a scene. When compared to frame-based counterparts, NVS enhances scene interpretation, processing time, reaction time, and power consumption. Deep-learning reconstruction networks are evaluated in this study to determine the applicability of existing state-of-the-art multi-modal image fusion techniques with the addition of NVS data rather than RGB data. As a benchmark, metrics such as signal-to-noise ratio (SNR) and pixel wise error are used.

show abstract