Attention-Guided Unified Network for Panoptic Segmentation

Li, Yanwei; Chen, Xinze; Zhu, Zheng; Xie, Lingxi; Huang, Guan; Du, Dalong; Wang, Xingang

doi:10.1109/cvpr.2019.00719

Cited by 252 publications

(172 citation statements)

References 53 publications

Supporting

Mentioning

171

Contrasting

Unclassified

Order By: Relevance

“…De Gaus et al [10] uses a single feature extraction backbone for the pyramid semantic segmentation head [39], and the instance segmentation head [15], followed by heuristics for merging pixel level annotations, effectively introducing an end-to-end version of [17] due to the shared backbone for the two task networks. Li et al [20] propose the attentionguided unified network (AUNet) which leverages proposal and mask level attention to better segment the background. Similar post-processing heuristics as in [17] are used to generate the final panoptic segmentation.…”

Section: Related Workmentioning

confidence: 99%

UPSNet: A Unified Panoptic Segmentation Network

Xiong

Liao

Zhao

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

361

309

View full text Add to dashboard Cite

In this paper, we propose a unified panoptic segmentation network (UPSNet) for tackling the newly proposed panoptic segmentation task. On top of a single backbone residual network, we first design a deformable convolution based semantic segmentation head and a Mask R-CNN style instance segmentation head which solve these two subtasks simultaneously. More importantly, we introduce a parameter-free panoptic head which solves the panoptic segmentation via pixel-wise classification. It first leverages the logits from the previous two heads and then innovatively expands the representation for enabling prediction of an extra unknown class which helps better resolve the conflicts between semantic and instance segmentation. Additionally, it handles the challenge caused by the varying number of instances and permits back propagation to the bottom modules in an end-to-end manner. Extensive experimental results on Cityscapes, COCO and our internal dataset demonstrate that our UPSNet achieves stateof-the-art performance with much faster inference.

show abstract

Section: Related Workmentioning

confidence: 99%

UPSNet: A Unified Panoptic Segmentation Network

Xiong

Liao

Zhao

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

361

309

View full text Add to dashboard Cite

show abstract

“…JSIS-Net [10] proposes a single network with the instance segmentation head [14] and the pyramid stuff segmentation head [48], following heuristics to merge two kinds of outputs. Li et al [23] propose AUNet that can leverage proposal and mask level attention and get better background results.…”

Section: Panoptic Segmentationmentioning

confidence: 99%

An End-To-End Network for Panoptic Segmentation

Liu

Peng²,

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

143

View full text Add to dashboard Cite

Panoptic segmentation, which needs to assign a category label to each pixel and segment each object instance simultaneously, is a challenging topic. Traditionally, the existing approaches utilize two independent models without sharing features, which makes the pipeline inefficient to implement. In addition, a heuristic method is usually employed to merge the results. However, the overlapping relationship between object instances is difficult to determine without sufficient context information during the merging process. To address the problems, we propose a novel end-to-end Occlusion Aware Network (OANet) for panoptic segmentation, which can efficiently and effectively predict both the instance and stuff segmentation in a single network. Moreover, we introduce a novel spatial ranking module to deal with the occlusion problem between the predicted instances. Extensive experiments have been done to validate the performance of our proposed method and promising results have been achieved on the COCO Panoptic benchmark.

show abstract

“…In [14], the method consists of a unified network similar to ours, as well as a consistency loss to make the output more consistent, but there is no additional information flow to boost the performance. AUNet [15] does leverage information exchange, but it requires complicated attention and masking operations. Our framework is designed to be simple and generally applicable, while leveraging the architecture by using additional information flow to improve the performance.…”

Section: Related Workmentioning

confidence: 99%

Single Network Panoptic Segmentation for Street Scene Understanding

Geus

Meletis

Dubbelman

2019

2019 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

In this work, we propose a single deep neural network for panoptic segmentation, for which the goal is to provide each individual pixel of an input image with a class label, as in semantic segmentation, as well as a unique identifier for specific objects in an image, following instance segmentation. Our network makes joint semantic and instance segmentation predictions and combines these to form an output in the panoptic format. This has two main benefits: firstly, the entire panoptic prediction is made in one pass, reducing the required computation time and resources; secondly, by learning the tasks jointly, information is shared between the two tasks, thereby improving performance. Our network is evaluated on two street scene datasets: Cityscapes and Mapillary Vistas. By leveraging information exchange and improving the merging heuristics, we increase the performance of the single network, and achieve a score of 23.9 on the Panoptic Quality (PQ) metric on Mapillary Vistas validation, with an input resolution of 640 x 900 pixels. On Cityscapes validation, our method achieves a PQ score of 45.9 with an input resolution of 512 x 1024 pixels. Moreover, our method decreases the prediction time by a factor of 2 with respect to separate networks.

show abstract

Attention-Guided Unified Network for Panoptic Segmentation

Cited by 252 publications

References 53 publications

UPSNet: A Unified Panoptic Segmentation Network

UPSNet: A Unified Panoptic Segmentation Network

An End-To-End Network for Panoptic Segmentation

Single Network Panoptic Segmentation for Street Scene Understanding

Contact Info

Product

Resources

About