Real-Time High-Performance Semantic Image Segmentation of Urban Street Scenes

Abstract

Deep Convolutional Neural Networks (DCNNs) have recently shown outstanding performance in semantic image segmentation. However, state-of-the-art DCNN-based semantic segmentation methods usually suffer from high computational complexity due to the use of complex network architectures. This greatly limits their applications in the real-world scenarios that require real-time processing. In this paper, we propose a real-time high-performance DCNN-based method for robust semantic segmentation of urban street scenes, which achieves a good trade-off between accuracy and speed. Specifically, a Lightweight Baseline Network with Atrous convolution and Attention (LBN-AA) is firstly used as our baseline network to efficiently obtain dense feature maps. Then, the Distinctive Atrous Spatial Pyramid Pooling (DASPP), which exploits the different sizes of pooling operations to encode the rich and distinctive semantic information, is developed to detect objects at multiple scales. Meanwhile, a Spatial detail-Preserving Network (SPN) with shallow convolutional layers is designed to generate highresolution feature maps preserving the detailed spatial information. Finally, a simple but practical Feature Fusion Network (FFN) is used to effectively combine both shallow and deep features from the semantic branch (DASPP) and the spatial branch (SPN), respectively. Extensive experimental results show that the proposed method respectively achieves the accuracy of 73.6% and 68.0% mean Intersection over Union (mIoU) with the inference speed of 51.0 fps and 39.3 fps on the challenging Cityscapes and CamVid test datasets (by only using a single NVIDIA TITAN X card). This demonstrates that the proposed method offers excellent performance at the real-time speed for semantic segmentation of urban street scenes.

PPANet: Point-Wise Pyramid Attention Network for Semantic Segmentation

“…Evaluation Metrics on KITTI. Precision and recall evaluation metrics can be considered one of the most common metrics for evaluating a binary classification; following the methods used in [64,66,67], we evaluated our segmentation model using precision Equation ( 7), recall Equation ( 8), and F-measure Equation (11). The evaluation metrics are listed in the following equations:…”

“…We have chosen some state-of-the-art models to perform a comparison with our proposed PPA-Net model. These models include SegNet [23], ENet [21], FastFCN [68], LBN-AA [66], DABNet [67], and AGLNet [69]. The overall results of PPANet and other SOTA models are illustrated in Figure 3.…”

“…From Table 4, we can see that the proposed PPANet method has superior performance in terms of mIoU. First, the model was compared with models that have been designed for real-time semantic segmentation such as ENet [21], BiSeNetv1 [70], CGNet [71], NDNet45-FCN8-LF [72], LBN-AA [66], DABNet [67], and AGLNet [69]. And also, we compared our proposed method with a non-realtime model such as DeebLabv2 [58], PSPNet [51], DenseDecoder [73], and SegNet [23].…”

AMSASeg: An Attention-Based Multi-Scale Atrous Convolutional Neural Network for Real-Time Object Segmentation From 3D Point Cloud

“…Image semantic segmentation have attracted interests of many researchers due to the wide applicability such as biomedical understanding and autonomous vehicles [8] - [15]. U-Net with an encoder-decoder architecture proposed in [8] successfully segmented medical images and presented a standard for segmentation architecture.…”

“…Meanwhile, authors in [15] considered pooling operation, which highly effected segmentation performance. They proposed an efficient pooling method to extract more distinctive features, namely distinctive atrous spatial pyramid pooling (DASPP).…”