Simultaneous Object Detection and Semantic Segmentation

Abstract

Both object detection in and semantic segmentation of camera images are important tasks for automated vehicles. Object detection is necessary so that the planning and behavior modules can reason about other road users. Semantic segmentation provides for example free space information and information about static and dynamic parts of the environment. There has been a lot of research to solve both tasks using Convolutional Neural Networks. These approaches give good results but are computationally demanding. In practice, a compromise has to be found between detection performance, detection quality and the number of tasks. Otherwise it is not possible to meet the real-time requirements of automated vehicles. In this work, we propose a neural network architecture to solve both tasks simultaneously. This architecture was designed to run with around 10 Hz on 1 MP images on current hardware. Our approach achieves a mean IoU of 61.2% for the semantic segmentation task on the challenging Cityscapes benchmark. It also achieves an average precision of 69.3% for cars and 67.7% on the moderate difficulty level of the KITTI benchmark.

Fusion of range measurements and semantic estimates in an evidential framework / Fusion von Distanzmessungen und semantischen Größen im Rahmen der Evidenztheorie

“…Its output is decoded to a semantic segmentation map with the original image resolution. The detailed architecture of this network is described in [7]. This neural network was trained on the Cityscapes dataset [2] by using both the 5k precisely and 20k coarsely annotated training images.…”

A Versatile Machine Vision Algorithm for Real-Time Counting Manually Assembled Pieces

“…In this work, we propose the use of the Motion Check method followed by the blob-based counting algorithm to build a simple, computationally fast, and yet very versatile solution which is able to count pieces, mostly based on color and morphology. We avoided testing very advanced MV solutions, based on Convolutional Neural Networks (CNN), for the counting algorithm, as they are too computationally intensive [ 28 , 29 ] and, given the mandatory necessity of our system to work in real-time, they likely require huge computational resources such as dedicated GPUs. Furthermore, the usage of a CNN is probably excessive in our context, as the environment is very restricted, and only hands and products interfere in the framed area.…”

Refining Semantic Segmentation with Superpixel by Transparent Initialization and Sparse Encoder

“…Semantic segmentation is an essential and challenging task in computer vision of a pixel-level mapping between predefined semantics and image pixels of interesting objects. It has various applications in the real world, such as autonomous vehicles (Feng et al 2020), object identification (Mottaghi et al 2014;Salscheider 2019), image editing (Hong et al 2018), scene analysis (Hofmarcher et al 2019), and so on. Recent developments of semantic segmentation are greatly promoted by deep learning on several large-scale datasets, such as Berkeley segmentation benchmark (Martin et al 2001) and the large-scale Microsoft Common Objects in Context 2017 (MS-COCO) (Lin et al 2014), resulting in effective networks including FCN (Long, Shelhamer, and Darrell 2015), PSP (Zhao et al 2017), CRFasRNN (Zheng et al 2015), DeepLab serial (Chen et al 2017(Chen et al , 2018a, etc.…”