DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentation

Abstract

This paper introduces an extremely efficient CNN architecture named DFANet for semantic segmentation under resource constraints. Our proposed network starts from a single lightweight backbone and aggregates discriminative features through sub-network and sub-stage cascade respectively. Based on the multi-scale feature propagation, DFANet substantially reduces the number of parameters, but still obtains sufficient receptive field and enhances the model learning ability, which strikes a balance between the speed and segmentation performance. Experiments on Cityscapes and CamVid datasets demonstrate the superior performance of DFANet with 8× less FLOPs and 2× faster than the existing state-of-the-art real-time semantic segmentation methods while providing comparable accuracy. Specifically, it achieves 70.3% Mean IOU on the Cityscapes test dataset with only 1.7 GFLOPs and a speed of 160 FPS on one NVIDIA Titan X card, and 71.3% Mean IOU with 3.4 GFLOPs while inferring on a higher resolution image.

BiSeNet: Bilateral Segmentation Network for Real-Time Semantic Segmentation

“…In the inference phase, we use the training dataset and validation dataset to train our model with 960 × 720 resolution input. Our models are compared to some non-real-time algorithms, including SegNet (Badrinarayanan et al, 2017), Deeplab (Chen et al, 2015), RTA (Huang et al, 2018), Dilate8 (Yu and Koltun, 2016), PSPNet (Zhao et al, 2017), VideoGCRF (Chandra et al, 2018), and DenseDecoder (Bilinski and Prisacariu, 2018), and real-time algorithms, containing ENet (Paszke et al, 2016), IC-Net (Zhao et al, 2018a), DABNet (Li et al, 2019a), DFANet (Li et al, 2019b), SwiftNet (Orsic et al, 2019), BiSeNetV1 (Yu et al, 2018a). BiSeNetV2 achieves much faster inference speed than other methods.…”

Deep multimodal fusion for semantic image segmentation: A survey

“…In addition to the aforementioned networks, many practical deep learning techniques (e.g., Spatial Pyramid Pooling [59], CRF-RNN [60], Batch Normalization [61], Dropout [62]) were proposed for improving the effectiveness of learning models. Notably, multi-scale feature aggregation was frequently used in semantic segmentation [63,64,65,66,67]. These learning models experimentally achieve significant performance improvement.…”

LDPNet: A Lightweight Densely Connected Pyramid Network for Real-Time Semantic Segmentation

“…Finally, a feature fusion module was designed to merge the two features effectively. A threelevel feature extraction network was designed by DFANet [18], which fully promoted the interaction and aggregation of feature information at different levels while making sure that the computational burden was small. On the other hand, a novel multi-feature fusion module was used by MSFNet [19] to strengthen the information flow among layers.…”

“…The batch size is 8, and the Adam optimizer is used to train the model with the weight decay of 1 × 10 −4 . Based on some previous work [15][16][17][18], we also adopt the "poly" learning rate strategy, as shown below:…”