Learning to see through the haze: Multi-sensor learning-fusion System for Vulnerable Traffic Participant Detection in Fog

Broughton, George; Majer, Filip; Rouček, Tomáš; Ruichek, Yassine; Yan, Zhi

doi:10.1016/j.robot.2020.103687

Cited by 16 publications

(6 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, while the experiments indicate that the technology is ready to be deployed in buildings or small residential clusters, complex urban scenarios require more advanced, socially-aware navigation (Kucner et al, 2017;Vintr et al, 2020), capable to deal with low visibility (Broughton et al, 2020). These capabilities have not been addressed in the proposed approach and will require further future investigation.…”

Section: Lesson Learnedmentioning

confidence: 97%

Design and Deployment of an Autonomous Unmanned Ground Vehicle for Urban Firefighting Scenarios

Jindal¹,

Wang²,

Thakur³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Autonomous mobile robots have the potential to solve missions that are either too complex or dangerous to be accomplished by humans. In this paper, we address the design and autonomous deployment of a ground vehicle equipped with a robotic arm for urban firefighting scenarios. We describe the hardware design and algorithm approaches for autonomous navigation, planning, fire source identification and abatement in unstructured urban scenarios. The approach employs on-board sensors for autonomous navigation and thermal camera information for source identification. A custom electro-mechanical pump is responsible to eject water for fire abatement. The proposed approach is validated through several experiments, where we show the ability to identify and abate a sample heat source in a building. The whole system was developed and deployed during the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020, for Challenge No. 3 -Fire Fighting Inside a High-Rise Building and during the Grand Challenge where our approach scored the highest number of points among all UGV solutions and was instrumental to win the first place.

show abstract

Section: Lesson Learnedmentioning

confidence: 97%

Design and Deployment of an Autonomous Unmanned Ground Vehicle for Urban Firefighting Scenarios

Jindal¹,

Wang²,

Thakur³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…During adverse weather conditions, a learning-based sensor fusion scheme was designed to make the system more rely on the radar to achieve higher accuracy of pedestrian detection and localization. George et al [57] further involved PointNet [75] in the online learning framework of sensor fusion to benefit from the current progress of deep learning methods of point cloud processing. PointNet was used to add a class label to each point of radar data, which was classified as clusters and finally sent to the sensor fusion module.…”

Section: E Sensor Fusionmentioning

confidence: 99%

“…It can undoubtedly save humans from tedious offline training tasks, and in mobile robotics, this learning paradigm further enables robots to learn on-site in their deployment place to adapt to changes in the environment. This paradigm has been used not only for LiDAR [36], [21], but also for millimeter wave radar with similar data representation [128], [57], not only for indoor service robots [36], but also for autonomous driving perception in urban environments [21].…”

Section: E Sensor Fusionmentioning

confidence: 99%

“…The success of Junior also directly affects and promotes the development of 3D LiDAR in autonomous driving [9], [6], [16], [17], [18], [19], [20], [21], [106], [59], [119], [108]. On the other hand, the demand for unconventional sensors typified by 3D LiDAR is also present in the field of service robots [14], [15], [36], [37], [87], [57], [56], [4], [54], [55], [52], [53]. Furthermore, in the face of the recent severe worldwide public health crisis, namely COVID-19, 3D LiDAR has also shown encouraging results and broad application prospects.…”

Section: E Sensor Fusionmentioning

confidence: 99%

See 1 more Smart Citation

3D ToF LiDAR in Mobile Robotics: A Review

Yang¹,

Li²,

Zhao³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In the past ten years, the use of 3D Time-of-Flight (ToF) LiDARs in mobile robotics has grown rapidly. Based on our accumulation of relevant research, this article systematically reviews and analyzes the use 3D ToF LiDARs in research and industrial applications. The former includes object detection, robot localization, long-term autonomy, LiDAR data processing under adverse weather conditions, and sensor fusion. The latter encompasses service robots, assisted and autonomous driving, and recent applications performed in response to public health crises. We hope that our efforts can effectively provide readers with relevant references and promote the deployment of existing mature technologies in real-world systems.

show abstract

“…Pioneering work in this field can be traced back more than two decades [19], in which a lifelong learning perspective for mobile robot control is presented. With the rapid development of various related technologies including hardware and algorithms, in recent years, research on robotic online learning has become more and more extensive [13], [20], [14], [15], [21], [22]. In particular, Teichman and Thrun [14] presented a semisupervised learning to the problem of track classification in 3D LiDAR data based on Expectation Maximization (EM) algorithm, which illustrated that learning of dynamic objects can benefit from tracking system.…”

Section: Related Workmentioning

confidence: 99%

Efficient Online Transfer Learning for 3D Object Classification in Autonomous Driving

Yang¹,

Yan²,

Tao³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Autonomous driving has achieved rapid development over the last few decades, including the machine perception as an important issue of it. Although object detection based on conventional cameras has achieved remarkable results in 2D/3D, non-visual sensors such as 3D LiDAR still have incomparable advantages in the accuracy of object position detection. However, the challenge also exists with the difficulty in properly interpreting point cloud generated by LiDAR. This paper presents a multi-modal-based online learning system for 3D LiDAR-based object classification in urban environments, including cars, cyclists and pedestrians. The proposed system aims to effectively transfer the mature detection capabilities based on visual sensors to the new model learning based on non-visual sensors through a multi-target tracker (i.e. using one sensor to train another). In particular, it integrates the Online Random Forests (ORF) [1] method, which inherently has the abilities of fast and multi-class learning. Through experiments, we show that our system is capable of learning a high-performance model for LiDAR-based 3D object classification on-the-fly, which is especially suitable for robotics in-situ deployment while responding to the widespread challenge of insufficient detector generalization capabilities.

show abstract

Learning to see through the haze: Multi-sensor learning-fusion System for Vulnerable Traffic Participant Detection in Fog

Cited by 16 publications

References 20 publications

Design and Deployment of an Autonomous Unmanned Ground Vehicle for Urban Firefighting Scenarios

Design and Deployment of an Autonomous Unmanned Ground Vehicle for Urban Firefighting Scenarios

3D ToF LiDAR in Mobile Robotics: A Review

Efficient Online Transfer Learning for 3D Object Classification in Autonomous Driving

Contact Info

Product

Resources

About