Graph-Guided Architecture Search for Real-Time Semantic Segmentation

Lin, Pei‐Wen; Sun, Peng; Cheng, Guangliang; Xie, Sirui; Li, Xi; Shi, Jianping

doi:10.1109/cvpr42600.2020.00426

Cited by 76 publications

(42 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, new approaches are focused on automatically obtaining a network architecture that is optimized for the specific case of interest. In Lin, Sun, Cheng, Xie, Li, & Shi (2020), this approach is studied in the case of semantic segmentation, obtaining good results in terms of both accuracy and inference time.…”

Section: Discussionmentioning

confidence: 99%

Multiclass semantic segmentation for digitisation of movable heritage using deep learning techniques

Patrucco

Setragno²

2021

Virtual archaeol. rev.

View full text Add to dashboard Cite

Digitisation processes of movable heritage are becoming increasingly popular to document the artworks stored in our museums. A growing number of strategies for the three-dimensional (3D) acquisition and modelling of these invaluable assets have been developed in the last few years. Their objective is to efficiently respond to this documentation need and contribute to deepening the knowledge of the masterpieces investigated constantly by researchers operating in many fieldworks. Nowadays, one of the most effective solutions is represented by the development of image-based techniques, usually connected to a Structure-from-Motion (SfM) photogrammetric approach. However, while images acquisition is relatively rapid, the processes connected to data processing are very time-consuming and require the operator’s substantial manual involvement. Developing deep learning-based strategies can be an effective solution to enhance the automatism level. In this research, which has been carried out in the framework of the digitisation of a wooden maquettes collection stored in the ‘Museo Egizio di Torino’, using a photogrammetric approach, an automatic masking strategy using deep learning techniques is proposed, to increase the level of automatism and therefore, optimise the photogrammetric pipeline. Starting from a manually annotated dataset, a neural network was trained to automatically perform a semantic classification to isolate the maquettes from the background. The proposed methodology allowed the researchers to obtain automatically segmented masks with a high degree of accuracy. The workflow is described (as regards acquisition strategies, dataset processing, and neural network training). In addition, the accuracy of the results is evaluated and discussed. Finally, the researchers proposed the possibility of performing a multiclass segmentation on the digital images to recognise different object categories in the images, as well as to define a semantic hierarchy to perform automatic classification of different elements in the acquired images.Highlights:<ul><li>In the framework of movable heritage digitisation processes, many procedures are very time-consuming, and they still require the operator’s substantial manual involvement.</li><li>This research proposes using deep learning techniques to enhance the automatism level in the generation of exclusion masks, improving the optimisation of the photogrammetric procedures.</li><li>Following this strategy, the possibility of performing a multiclass semantic segmentation (on the 2D images and, consequently, on the 3D point cloud) is also discussed, considering the accuracy of the obtainable results.</li></ul>

show abstract

Section: Discussionmentioning

confidence: 99%

Multiclass semantic segmentation for digitisation of movable heritage using deep learning techniques

Patrucco

Setragno²

2021

Virtual archaeol. rev.

View full text Add to dashboard Cite

show abstract

“…The Cityscapes dataset is a large-scale and high-quality semantic segmentation dataset, which consists of 5000 densely-annotated images collected on urban street. The resolution of the image in this dataset is 1024×2048 and each pixel is annotated with one of 19 Table 1 presents the quantitative results of different approaches achieved on the Cityscape test set, including the very recent DABNet [23], FPENet [24], DFANet A [22], BiSeNet [21], CAS [25], and GAS [26]. For a fair comparison of the FPS during inference, we also listed the input image size and the hardware specifications (i.e.…”

Section: Semantic Segmentation On Public Benchmarksmentioning

confidence: 99%

Cgan-Net: Class-Guided Asymmetric Non-Local Network for Real-Time Semantic Segmentation

Chen

Yang

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

By introducing various non-local blocks to capture the longrange dependencies, remarkable progress has been achieved in semantic segmentation recently. However, the improvement in segmentation accuracy usually comes at the price of significant reductions in network efficiency, as non-local block usually requires expensive computation and memory cost for dense pixel-to-pixel correlation. In this paper, we introduce a Class-Guided Asymmetric Non-local Network (CGAN-Net) to enhance the class-discriminability in learned feature map, while maintaining real-time efficiency.The key to our approach is to calculate the dense similarity matrix in coarse semantic prediction maps, instead of the high-dimensional latent feature map. This is not only computationally and memory efficient, but helps to learn query-dependent global context. Experiments conducted on Cityscape and CamVid demonstrate the compelling performance of our CGAN-Net. In particular, our network achieves 76.8% mean IoU on the Cityscapes test set with a speed of 38 FPS for 1024×2048 images on a single Tesla V100 GPU.

show abstract

“…With the rise of deep learning, self-driving cars have become increasingly popular in recent years. Semantic segmentation task is one of the fundamental and challenging tasks in computer vision and deep learning, and it can be widely applied to different applications, such as autonomous driving [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 ] and video surveillance systems [ 11 , 12 ], accompanying sensors on vehicles. Different from the general classification [ 13 , 14 , 15 , 16 ] and object detection [ 17 , 18 , 19 ] tasks, with semantic segmentation task it is more difficult to achieve good results because it needs to assign a label to each pixel in an image.…”

Section: Introductionmentioning

confidence: 99%

“…In addition, semantic segmentation task can be differentiated into real-time semantic segmentation [ 6 , 7 , 8 , 9 , 10 ] and non-real-time semantic segmentation [ 1 , 2 , 3 , 4 , 5 ]. The goal of real-time semantic segmentation is to find the best trade-off among the accuracy, inference time, and the number of parameters in the network.…”

Section: Introductionmentioning

confidence: 99%

“…The number of parameters of the network in non-real-time semantic segmentation is at least 10 times or even 100 times that of the number of parameters of the model in real-time semantic segmentation. Although several previous methods proposed extreme lightweight [ 21 , 22 ] or fast inference time [ 6 , 7 , 10 ] models for real-time semantic segmentation, the accuracy of these methods have a large gap from the accuracy of non-real-time semantic segmentation, and they may classify most of the objects incompletely.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Real-Time Semantic Segmentation with Dual Encoder and Self-Attention Mechanism for Autonomous Driving

Chang

Tsai

Lin

et al. 2021

Sensors

View full text Add to dashboard Cite

As the techniques of autonomous driving become increasingly valued and universal, real-time semantic segmentation has become very popular and challenging in the field of deep learning and computer vision in recent years. However, in order to apply the deep learning model to edge devices accompanying sensors on vehicles, we need to design a structure that has the best trade-off between accuracy and inference time. In previous works, several methods sacrificed accuracy to obtain a faster inference time, while others aimed to find the best accuracy under the condition of real time. Nevertheless, the accuracies of previous real-time semantic segmentation methods still have a large gap compared to general semantic segmentation methods. As a result, we propose a network architecture based on a dual encoder and a self-attention mechanism. Compared with preceding works, we achieved a 78.6% mIoU with a speed of 39.4 FPS with a 1024 × 2048 resolution on a Cityscapes test submission.

show abstract

Graph-Guided Architecture Search for Real-Time Semantic Segmentation

Cited by 76 publications

References 21 publications

Multiclass semantic segmentation for digitisation of movable heritage using deep learning techniques

Multiclass semantic segmentation for digitisation of movable heritage using deep learning techniques

Cgan-Net: Class-Guided Asymmetric Non-Local Network for Real-Time Semantic Segmentation

Real-Time Semantic Segmentation with Dual Encoder and Self-Attention Mechanism for Autonomous Driving

Contact Info

Product

Resources

About