Abstract. Camera based navigation in dynamic environments with high content of moving objects is challenging. Keypoint-based localization methods need to reliably reject features that do not belong to the static background. Here, traditional statistical methods for outlier rejection quickly reach their limits. A common approach is the combination with an inertial measurement unit for visual-inertial odometry. Also, deep learning based semantic segmentation was recently successfully applied in camera based localization to identify features on common objects. In this work, we study the application of mask-based feature selection based on semantic segmentation for robust localization in high dynamic environments. We focus on visual-inertial odometry, but similarly investigate a state-of-the-art pure vision-based method as baseline. For a versatile evaluation, we use challenging self-recorded datasets based on different sensor systems. This includes a combined dataset of a real world system and its synthetic clone with a large number of humans for in-depth analysis. We further deploy large-scale datasets from pedestrian navigation in a mall with escalator scenes and vehicle navigation during the day and at night. Our results show that visual-inertial odometry performs generally well in dynamic environments itself, but also shows significant failures in challenging scenes, which are prevented by using the segmentation aid.
Research from the field of planetary- and geoscience require investigation of geological features in harsh environments. Sampled data needs to be precisely localized spatially and in time, and to be prepared appropriately for further inspection and evaluation. In the case of thermal image data, the computer vision community has made enormous progress in the past years to combine thermal with spatial information in form of thermal 3D models. In this paper, we propose to use a camera-based hand-held sensor system to capture and georeference thermal images in rugged terrain and to prepare the data for further investigations by visualizing the thermal data in 3D. We use a global localization solution to gain fast 3D impressions of the environment and further Structure from Motion to generate detailed mid-scale models of selected interesting structures. We demonstrate our application based on a challenging dataset that we acquired in the active fumarole fields of Vulcano, Sicily.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.