Keyframe-based Direct Thermal–Inertial Odometry

Abstract

This paper proposes an approach for fusing direct radiometric data from a thermal camera with inertial measurements to extend the robotic capabilities of aerial robots for navigation in GPS-denied and visually degraded environments in the conditions of darkness and in the presence of airborne obscurants such as dust, fog and smoke. An optimization based approach is developed that jointly minimizes the re-projection error of 3D landmarks and inertial measurement errors. The developed solution is extensively verified against both groundtruth in an indoor laboratory setting, as well as inside an underground mine under severely visually degraded conditions.

Keyframe‐based thermal–inertial odometry

“…The real‐world application and performance is then verified and demonstrated through a set of field experiments inside both active and abandoned subterranean mines and in the context of navigation and full autonomous exploration missions, as well as an experiment inside an urban environment. Overall, and in terms of extending our previous efforts in Khattak et al (), the presented work makes the following additional contributions: A detailed presentation of every algorithmic step of the proposed KTIO, including elements not presented in Khattak et al () such as the marginalization policy. The thorough field evaluation of our method by utilizing it for autonomous aerial exploration missions in different underground mines, alongside an automated path inside an urban parking lot, showing the robustness and applicability of KTIO for diverse GPS‐denied and visually degraded environments. The different experiments involve different cases of visual degradation, geometries, thermal context, flight profiles, and in one case a different thermal camera sensor.…”

“…Furthermore, inertial measurements are incorporated in a joint optimization scheme to make the odometry estimation process more robust during cases of large motion. The proposed approach, which extends our previous efforts (Khattak, Papachristos, & Alexis, ), is thoroughly evaluated by first utilizing it to estimate the odometry of an aerial robot, flying in a completely dark environment, and comparing it against ground‐truth. The real‐world application and performance is then verified and demonstrated through a set of field experiments inside both active and abandoned subterranean mines and in the context of navigation and full autonomous exploration missions, as well as an experiment inside an urban environment.…”

“…A detailed presentation of every algorithmic step of the proposed KTIO, including elements not presented in Khattak et al () such as the marginalization policy.…”

Motion Primitives-based Path Planning for Fast and Agile Exploration using Aerial Robots

“…The proposed planner subscribes to the data provided by the localization and mapping solution and provides references to the onboard MPC. Details for the overall system solution can be found in [2,3,26,27]. The depth sensor integrated on the platform is a Velodyne PuckLITE which provides a horizontal and vertical field of view of F H = 360 • , F V = 30 • .…”

“…Research in autonomous robotic exploration and mapping of unknown environments is expanding into an ever increasing set of application domains. Pushing the frontier with respect to the settings and environments within which robots can be utilized as explorers [1][2][3][4][5][6], first responders [7,8], and inspectors [9][10][11][12], aerial vehicles, in particular, are currently employed in a multitude of civilian and military applications. Nevertheless, despite the unprecedented progress in the domain and the multiple exploration strategies proposed [13][14][15][16][17][18][19][20][21], the current state-of-the-art, as demonstrated experimentally, is limited to rather low-speed conservative missions as the robots try to guarantee safe navigation and simultaneous optimized selection of subsequent exploration moves given their real-time onboard localization and mapping capabilities.…”