Season-Invariant Semantic Segmentation with a Deep Multimodal Network

Kim, Yon Su; Maturana, Daniel; Uenoyama, Masashi; Scherer, Sebastian

doi:10.1007/978-3-319-67361-5_17

Cited by 25 publications

(25 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many works [61], [91], [99]- [101], [106], [108], [109], [111], [117]- [120], [123] deal with the 2D object detection problem on the front-view 2D image plane. Compared to 2D detection, 3D detection is more challenging since the object's distance to the ego-vehicle needs to be estimated.…”

Section: ) 2d or 3d Detectionmentioning

confidence: 99%

Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges

Feng

Haase-Schütz

Rosenbaum

et al. 2021

IEEE Trans. Intell. Transport. Syst.

793

338

View full text Add to dashboard Cite

Recent advancements in perception for autonomous driving are driven by deep learning. In order to achieve robust and accurate scene understanding, autonomous vehicles are usually equipped with different sensors (e.g. cameras, LiDARs, Radars), and multiple sensing modalities can be fused to exploit their complementary properties. In this context, many methods have been proposed for deep multi-modal perception problems. However, there is no general guideline for network architecture design, and questions of "what to fuse", "when to fuse", and "how to fuse" remain open. This review paper attempts to systematically summarize methodologies and discuss challenges for deep multi-modal object detection and semantic segmentation in autonomous driving. To this end, we first provide an overview of on-board sensors on test vehicles, open datasets, and background information for object detection and semantic segmentation in autonomous driving research. We then summarize the fusion methodologies and discuss challenges and open questions. In the appendix, we provide tables that summarize topics and methods. We also provide an interactive online platform to navigate each reference: https://boschresearch.github.io/multimodalperception/. 0.99 0.8 0.98 0.99 0.96 0.96 0.94 Vehicle Person Road sign Traffic light LiDAR Points Map Radar Points RGB Image

show abstract

Section: ) 2d or 3d Detectionmentioning

confidence: 99%

Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges

Feng

Haase-Schütz

Rosenbaum

et al. 2021

IEEE Trans. Intell. Transport. Syst.

793

338

View full text Add to dashboard Cite

show abstract

“…Although other off-road datasets exist in the literature [21], [22], one predominant publicly available off-road dataset is the Freiburg Forest dataset [6]. However, one critical problem with this dataset is that the images are not of the same size, suggesting inconsistencies in its pre-processing.…”

Section: Related Work a Off-road Datasets And Challengesmentioning

confidence: 99%

Multi-Resolution and Multi-Domain Analysis of Off-Road Datasets for Autonomous Driving

Mayuku

Surgenor

Marshall

2021

2021 18th Conference on Robots and Vision (CRV)

View full text Add to dashboard Cite

For use in off-road autonomous driving applications, we propose and study the use of multi-resolution local binary pattern texture descriptors to improve overall semantic segmentation performance and reduce class imbalance effects in off-road visual datasets. Our experiments, using a challenging publicly available off-road dataset as well as our own off-road dataset, show that texture features provide added flexibility towards reducing class imbalance effects, and that fusing color and texture features can improve segmentation performance. Finally, we demonstrate domain adaptation limitations in nominally similar off-road environments by cross-comparing the segmentation performance of convolutional neural networks trained on both datasets.

show abstract

“…Nature features within off-road areas lead to particular problems to be solved in object detection domain (Dong-Ki Kim et al, 2017). Even a term "off-road area" is not defined distinctly.…”

Section: Special Conditionsmentioning

confidence: 99%

Semantic Scene Understanding for the Autonomous Platform

Vishnyakov

Blokhinov

Sgibnev

et al. 2020

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. In this paper we describe a new multi-sensor platform for data collection and algorithm testing. We propose a couple of methods for solution of semantic scene understanding problem for land autonomous vehicles. We describe our approaches for automatic camera and LiDAR calibration; three-dimensional scene reconstruction and odometry calculation; semantic segmentation that provides obstacle recognition and underlying surface classification; object detection; point cloud segmentation. Also, we describe our virtual simulation complex based on Unreal Engine, that can be used for both data collection and algorithm testing. We collected a large database of field and virtual data: more than 1,000,000 real images with corresponding LiDAR data and more than 3,500,000 simulated images with corresponding LiDAR data. All proposed methods were implemented and tested on our autonomous platform; accuracy estimates were obtained on the collected database.

show abstract

Season-Invariant Semantic Segmentation with a Deep Multimodal Network

Cited by 25 publications

References 15 publications

Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges

Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges

Multi-Resolution and Multi-Domain Analysis of Off-Road Datasets for Autonomous Driving

Semantic Scene Understanding for the Autonomous Platform

Contact Info

Product

Resources

About