Sparse Depth Enhanced Direct Thermal-Infrared SLAM Beyond the Visible Spectrum

Shin, Young-Sik; Kim, Ayoung

doi:10.1109/lra.2019.2923381

Cited by 49 publications

(24 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent works on large-scale and real-time visual SLAM show high-quality maps and loop-closing in outdoor urban environments (Lynen et al, 2020; Tanner et al, 2020), where GPS and 3D lidar are used as ground truth. Moreover, works in urban environments using vision, semantic mapping (Cadena et al, 2016), and including TIR images (Shin and Kim, 2019) offer promising techniques for potential SLAM applications in disaster robotics. Nevertheless, semantic segmentation is a recent research area whose attention is currently focused on indoor (Milioto and Stachniss, 2019) and urban environments (Zhang et al, 2018), but could be an interesting tool for disaster scenarios (Jeon et al, 2020).…”

Section: Discussionmentioning

confidence: 99%

The UMA-SAR Dataset: Multimodal data collection from a ground vehicle during outdoor disaster response training exercises

Morales

Vázquez-Martín

Mandow

et al. 2021

The International Journal of Robotics Research

View full text Add to dashboard Cite

This article presents a collection of multimodal raw data captured from a manned all-terrain vehicle in the course of two realistic outdoor search and rescue (SAR) exercises for actual emergency responders conducted in Málaga (Spain) in 2018 and 2019: the UMA-SAR dataset. The sensor suite, applicable to unmanned ground vehicles (UGVs), consisted of overlapping visible light (RGB) and thermal infrared (TIR) forward-looking monocular cameras, a Velodyne HDL-32 three-dimensional (3D) lidar, as well as an inertial measurement unit (IMU) and two global positioning system (GPS) receivers as ground truth. Our mission was to collect a wide range of data from the SAR domain, including persons, vehicles, debris, and SAR activity on unstructured terrain. In particular, four data sequences were collected following closed-loop routes during the exercises, with a total path length of 5.2 km and a total time of 77 min. In addition, we provide three more sequences of the empty site for comparison purposes (an extra 4.9 km and 46 min). Furthermore, the data is offered both in human-readable format and as rosbag files, and two specific software tools are provided for extracting and adapting this dataset to the users’ preference. The review of previously published disaster robotics repositories indicates that this dataset can contribute to fill a gap regarding visual and thermal datasets and can serve as a research tool for cross-cutting areas such as multispectral image fusion, machine learning for scene understanding, person and object detection, and localization and mapping in unstructured environments. The full dataset is publicly available at: www.uma.es/robotics-and-mechatronics/sar-datasets .

show abstract

Section: Discussionmentioning

confidence: 99%

The UMA-SAR Dataset: Multimodal data collection from a ground vehicle during outdoor disaster response training exercises

Morales

Vázquez-Martín

Mandow

et al. 2021

The International Journal of Robotics Research

View full text Add to dashboard Cite

show abstract

“…Furthermore, since most of the SSLs are manufactured with Micro-electromechanical Systems (MEMS), the optical mechanism restricted their field-of-view (FOV) [ 43 ]. Using Velodyne Velarray M1600 as an example, the sensor only has a 120° horizontal FOV and 35° vertical FOV.…”

Section: Multi-lidar Mapping Overviewmentioning

confidence: 99%

Enhancing Solid State LiDAR Mapping with a 2D Spinning LiDAR in Urban Scenario SLAM on Ground Vehicles

Wei

Shirinzadeh

Nowell

et al. 2021

Sensors

View full text Add to dashboard Cite

Solid-State LiDAR (SSL) takes an increasing share of the LiDAR market. Compared with traditional spinning LiDAR, SSLs are more compact, energy-efficient and cost-effective. Generally, the current study of SSL mapping is limited to adapting existing SLAM algorithms to an SSL sensor. However, compared with spinning LiDARs, SSLs are different in terms of their irregular scan patterns and limited FOV. Directly applying existing SLAM approaches on them often increase the instability of a mapping process. This study proposes a systematic design, which consists of a dual-LiDAR mapping system and a three DOF interpolated six DOF odometry. For dual-LiDAR mapping, this work uses a 2D LiDAR to enhance a 3D SSL performance on a ground vehicle platform. The proposed system takes a 2D LiDAR to preprocess the scanning field into a number of feature sections according to the curvatures on the 2D fraction. Subsequently, this section information is passed to 3D SSL for direction feature selection. Additionally, this work proposes an odometry interpolation method which uses both LiDARs to generate two separated odometries. The proposed odometry interpolation method selectively determines the appropriate odometry information to update the system state under challenging conditions. Experiments are conducted in different scenarios. The results proves that the proposed approach is able to utilise 12 times more corner features from the environment than the comparied method, thus results in a demonstrable improvement in its absolute position error.

show abstract

“…We use two embedded Nvidia TX2 to balance the processing load of multi-cameras, implement efficient front end visual odometer strategies. The flow chart of related frame management and pose management is shown in the figure 10 For poses management, the example we presented in figure 9 will be detailed. For infrared stereos, frame synchronization is deployed.…”

Section: Frames and Poses Managementmentioning

confidence: 99%

“…[9] combine RGB information and thermal feature together, use back-end optimization with loop closure, update map and location. [10] cooperated thermal-infrared camera with LiDAR for density map construction. All these works use the thermal-infrared camera as complementary of other sensors.…”

Section: Introductionmentioning

confidence: 99%

Large Field of View Cooperative Infrared and Visible Spectral Sensor for Visual Odometry

Wang

Yang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Thermal images produced by the long-wavelength infrared (LWIR) camera are robust and independent from environmental illumination change. They can help the standard visible light camera working under the complicated environmental condition. Breaking through the traditional stereo multi-spectral sensor consisting of a visible-light camera and a LWIR camera, a novel architecture of large field of view (FOV) cooperated infrared and visible spectral sensor for visual odometry is proposed. The novel sensor is equipped with two visible cameras, four infrared cameras covering 120 degrees FOV in horizontal under both bands. Distribution of cameras and related peripheral devices are specifically designed which makes the sensor's volume less than 100 cm (length) × 10 cm (height) × 10 cm (width). The sensor's cameras calibration, distortion correction and measurement principle are elaborated. Feature-based method for visible and multi-windowed optimization-based image alignment for infrared is designed for the visual odometer based on the different imaging mechanism and distribution of cameras in the sensor. The frames and estimated poses management from both bands are proposed. Moreover, all proposed methodologies can be implemented in the sensor's embedded processor. The electrical power consumption is only 12W. Experiments of the sensor's evaluation are performed, experimental results show that large FOV cooperated multi-spectral cameras can efficiently improve the robustness of visual odometry. The real-time performance of the sensor is higher than 10fps with disparity map construction under both bands.

show abstract

Sparse Depth Enhanced Direct Thermal-Infrared SLAM Beyond the Visible Spectrum

Cited by 49 publications

References 33 publications

The UMA-SAR Dataset: Multimodal data collection from a ground vehicle during outdoor disaster response training exercises

The UMA-SAR Dataset: Multimodal data collection from a ground vehicle during outdoor disaster response training exercises

Enhancing Solid State LiDAR Mapping with a 2D Spinning LiDAR in Urban Scenario SLAM on Ground Vehicles

Large Field of View Cooperative Infrared and Visible Spectral Sensor for Visual Odometry

Contact Info

Product

Resources

About