Multi-Modal Sensor Fusion-Based Semantic Segmentation for Snow Driving Scenarios

Vachmanus, Sirawich; Ravankar, Ankit A.; Emaru, Takanori; Kobayashi, Yukinori

doi:10.1109/jsen.2021.3077029

Cited by 26 publications

(7 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, by using a deep learning model to fuse the data streams emanating from a visual camera and a thermal sensor, semantic segmentation can be performed in snowy weather. 13 The vision-thermal fusion model detects persons with higher accuracy than a camera-only model in snowy scenarios. This example demonstrates that diverse sensor inputs can be used to compensate for various unsuitable weather conditions in the self-driving context.…”

Section: Self-driving Applicationsmentioning

confidence: 99%

“…One example of deep-learning -based satellite fusion is superresolution imagery, for which images from Landsat and Sentinel-2 are fused, and time series images with high resolution are generated. 13 Another example is the fusion of satellite images from homogeneous satellites but in different modes for detecting ice-wedge polygons across the Arctic region. 14 Superior Earth observations enable scientists to closely monitor Earth in real time and precisely model planetary changes to generate predictions.…”

Section: Satellite Applications Assisted By Remote Sensing Technologymentioning

confidence: 99%

See 1 more Smart Citation

Staying Stronger Together: Synergy From Multiple Sensors

Lin

2022

Computer

View full text Add to dashboard Cite

Section: Self-driving Applicationsmentioning

confidence: 99%

Section: Satellite Applications Assisted By Remote Sensing Technologymentioning

confidence: 99%

Staying Stronger Together: Synergy From Multiple Sensors

Lin

2022

Computer

View full text Add to dashboard Cite

“…Histogram of oriented gradient (HOG) and autocorrelation loss are used to facilitate the orientation consistency and repress repetitive rain streaks. They trained the network all the way from drizzle to downpour rain Fusion [110] LiDAR [152] LiDAR [76] LiDAR [153] Others [154] LiDAR [155] LiDAR [156] Camera [157] Camera [158] Camera [159] Camera [160] Camera [161] Camera [162] Camera [163] Camera [164] LiDAR [165] LiDAR [166] LiDAR [128] LiDAR [29] Fusion [129] LiDAR [167] Fusion [168] LiDAR [169] LiDAR [170] Fusion [171] LiDAR [172] Camera [173] Camera [174] Camera [175] Camera [176] Camera [177] Camera [178] Camera [179] Camera [180] Camera [181] Camera [182] Camera [183] Camera [184] Camera [185] Camera [186] Fusion [187] Fusion [188] LiDAR [189] Camera [190] Camera…”

Section: Rainmentioning

confidence: 99%

“…Vachmanus et al [188] extended this idea into the autonomous driving semantic segmentation task by adding thermal cameras into the modality. RGB camera input might not be enough to represent every pertinent object with various colors in the surroundings, or pedestrians involved in the snow driving scenario, which happens to be the thermal camera's strong point.…”

Section: Snowmentioning

confidence: 99%

Perception and Sensing for Autonomous Vehicles Under Adverse Weather Conditions: A Survey

Zhang¹,

Carballo²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Automated Driving Systems (ADS) open up a new domain for the automotive industry and offer new possibilities for future transportation with higher efficiency and comfortable experiences. However, autonomous driving under adverse weather conditions has been the problem that keeps autonomous vehicles (AVs) from going to level 4 or higher autonomy for a long time. This paper assesses the influences and challenges that weather brings to ADS sensors in an analytic and statistical way, and surveys the solutions against inclement weather conditions. Stateof-the-art techniques on perception enhancement with regard to each kind of weather are thoroughly reported. External auxiliary solutions like V2X technology, weather conditions coverage in currently available datasets, simulators, and experimental facilities with weather chambers are distinctly sorted out. By pointing out all kinds of major weather problems the autonomous driving field is currently facing, and reviewing both hardware and computer science solutions in recent years, this survey contributes a holistic overview on the obstacles and directions of ADS development in terms of adverse weather driving conditions.

show abstract

“…Multimodal sensors for mobile agents only focus on indoor localization [12]. Studies to overcome the challenges of the outdoor environment, such as snow, help autonomous driving by learning with color and infrared images; however, it seems insufficient to recognize abnormal situations in outdoor monitoring areas [13]. Multimodal sensors for object detection in marine environments use color-based learning and can only be used during the daytime [14].…”

Section: Introductionmentioning

confidence: 99%

Design of Multimodal Sensor Module for Outdoor Robot Surveillance System

et al. 2022

View full text Add to dashboard Cite

Recent studies on surveillance systems have employed various sensors to recognize and understand outdoor environments. In a complex outdoor environment, useful sensor data obtained under all weather conditions, during the night and day, can be utilized for application to robots in a real environment. Autonomous surveillance systems require a sensor system that can acquire various types of sensor data and can be easily mounted on fixed and mobile agents. In this study, we propose a method for modularizing multiple vision and sound sensors into one system, extracting data synchronized with 3D LiDAR sensors, and matching them to obtain data from various outdoor environments. The proposed multimodal sensor module can acquire six types of images: RGB, thermal, night vision, depth, fast RGB, and IR. Using the proposed module with a 3D LiDAR sensor, multimodal sensor data were obtained from fixed and mobile agents and tested for more than four years. To further prove its usefulness, this module was used as a monitoring system for six months to monitor anomalies occurring at a given site. In the future, we expect that the data obtained from multimodal sensor systems can be used for various applications in outdoor environments.

show abstract

Multi-Modal Sensor Fusion-Based Semantic Segmentation for Snow Driving Scenarios

Cited by 26 publications

References 47 publications

Staying Stronger Together: Synergy From Multiple Sensors

Staying Stronger Together: Synergy From Multiple Sensors

Perception and Sensing for Autonomous Vehicles Under Adverse Weather Conditions: A Survey

Design of Multimodal Sensor Module for Outdoor Robot Surveillance System

Contact Info

Product

Resources

About