PolarMask: Single Shot Instance Segmentation with Polar Representation

Xie, Enze; Sun, Peize; Song, Xiaoge; Wang, Wenhai; Ding, Liang; Shen, Chunhua; Luo, Ping

doi:10.48550/arxiv.1909.13226

Cited by 31 publications

(30 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(2) The proposed method has a huge superiority on rotated object detection and cell instance segmentation, and the runtime speed is fast. (3) Compared to the prior PolarMask [61], the proposed method achieves much better performance, which strongly proves the effectiveness of soft polar centerness and refined feature pyramid.…”

Section: Summarizing the Experimental Resultsmentioning

confidence: 67%

“…We show that the proposed IoU loss in polar space can largely ease the optimization and improve accuracy, compared with the standard loss such as the smooth-ł1 loss. In parallel, soft polar centerness improves the previous centreness loss in FCOS [56] and PolarMask [61], leading to further boost in performance. (3) Rich experiments show that state-of-the-art performances of object instance segmentation and rotated object detection can be achieved with low computational overhead in multiple challenging benchmarks.…”

Section: Arxiv:210502184v1 [Cscv] 5 May 2021mentioning

confidence: 98%

“…In PolarMask [61], the Polar CenterNess is proposed to re-weight positive samples. The comparisons are shown in Table 4c.…”

Section: Centerness Strategymentioning

confidence: 99%

“…In PolarMask++, we further improve polar centerness [61] by proposing its soft version. From Table 4e, we can find that on the one hand, soft polar centerness can improve more than 0.6% AP compared to the original polar centerness.…”

Section: Centerness Strategymentioning

confidence: 99%

“…The feature pyramid network (FPN) is another key component in PolarMask [61]. As show in Table 4f, the original FPN can achieve 29.8% AP, while using PANet [39] can only improve 0.1% AP.…”

Section: Strategy Of Feature Pyramidmentioning

confidence: 99%

See 4 more Smart Citations

PolarMask++: Enhanced Polar Representation for Single-Shot Instance Segmentation and Beyond

Xie

Wang

Ding

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

Reducing the complexity of the pipeline of instance segmentation is crucial for real-world applications. This work addresses this issue by introducing an anchor-box free and single-shot instance segmentation framework, termed PolarMask, which reformulates the instance segmentation problem as predicting the contours of objects in the polar coordinate, with several appealing benefits. (1) The polar representation unifies instance segmentation (masks) and object detection (bounding boxes) into a single framework, reducing the design and computational complexity. (2) Two modules are carefully designed (i.e. soft polar centerness and polar IoU loss) to sample high-quality center examples and optimize polar contour regression, making the performance of PolarMask does not depend on the bounding box prediction results and thus becomes more efficient in training. (3) PolarMask is fully convolutional and can be easily embedded into most off-the-shelf detection methods. To further improve the accuracy of the framework, a Refined Feature Pyramid is introduced to further improve the feature representation at different scales, termed PolarMask++. Extensive experiments demonstrate the effectiveness of both PolarMask and PolarMask++, which achieve competitive results on instance segmentation in the challenging COCO dataset with single-model and single-scale training and testing, as well as new state-of-the-art results on rotate text detection and cell segmentation. We hope the proposed polar representation can provide a new perspective for designing algorithms to solve single-shot instance segmentation. The codes and models are available at: github.com/xieenze/PolarMask.

show abstract

Section: Summarizing the Experimental Resultsmentioning

confidence: 67%

Section: Arxiv:210502184v1 [Cscv] 5 May 2021mentioning

confidence: 98%

“…In PolarMask [61], the Polar CenterNess is proposed to re-weight positive samples. The comparisons are shown in Table 4c.…”

Section: Centerness Strategymentioning

confidence: 99%

Section: Centerness Strategymentioning

confidence: 99%

“…The feature pyramid network (FPN) is another key component in PolarMask [61]. As show in Table 4f, the original FPN can achieve 29.8% AP, while using PANet [39] can only improve 0.1% AP.…”

Section: Strategy Of Feature Pyramidmentioning

confidence: 99%

See 3 more Smart Citations

PolarMask++: Enhanced Polar Representation for Single-Shot Instance Segmentation and Beyond

Xie

Wang

Ding

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation

Cao

Anwer

Cholakkal

et al. 2020

Lecture Notes in Computer Science

120

159

View full text Add to dashboard Cite

Single-stage instance segmentation approaches have recently gained popularity due to their speed and simplicity, but are still lagging behind in accuracy, compared to two-stage methods. We propose a fast single-stage instance segmentation method, called SipMask, that preserves instance-specific spatial information by separating mask prediction of an instance to different sub-regions of a detected bounding-box. Our main contribution is a novel light-weight spatial preservation (SP) module that generates a separate set of spatial coefficients for each subregion within a bounding-box, leading to improved mask predictions. It also enables accurate delineation of spatially adjacent instances. Further, we introduce a mask alignment weighting loss and a feature alignment scheme to better correlate mask prediction with object detection. On COCO test-dev, our SipMask outperforms the existing single-stage methods. Compared to the state-of-the-art single-stage TensorMask, Sip-Mask obtains an absolute gain of 1.0% (mask AP), while providing a four-fold speedup. In terms of real-time capabilities, SipMask outperforms YOLACT with an absolute gain of 3.0% (mask AP) under similar settings, while operating at comparable speed on a Titan Xp. We also evaluate our SipMask for real-time video instance segmentation, achieving promising results on YouTube-VIS dataset. The source code is available at https://github.com/JialeCao001/SipMask.

show abstract

Point-Set Anchors for Object Detection, Instance Segmentation and Pose Estimation

Wei

Sun

et al. 2020

Lecture Notes in Computer Science

109

View full text Add to dashboard Cite

A recent approach for object detection and human pose estimation is to regress bounding boxes or human keypoints from a central point on the object or person. While this center-point regression is simple and efficient, we argue that the image features extracted at a central point contain limited information for predicting distant keypoints or bounding box boundaries, due to object deformation and scale/orientation variation. To facilitate inference, we propose to instead perform regression from a set of points placed at more advantageous positions. This point set is arranged to reflect a good initialization for the given task, such as modes in the training data for pose estimation, which lie closer to the ground truth than the central point and provide more informative features for regression. As the utility of a point set depends on how well its scale, aspect ratio and rotation matches the target, we adopt the anchor box technique of sampling these transformations to generate additional point-set candidates. We apply this proposed framework, called Point-Set Anchors, to object detection, instance segmentation, and human pose estimation. Our results show that this general-purpose approach can achieve performance competitive with state-of-the-art methods for each of these tasks. Code is available at https://github.com/FangyunWei/PointSetAnchor.

show abstract

PolarMask: Single Shot Instance Segmentation with Polar Representation

Cited by 31 publications

References 33 publications

PolarMask++: Enhanced Polar Representation for Single-Shot Instance Segmentation and Beyond

PolarMask++: Enhanced Polar Representation for Single-Shot Instance Segmentation and Beyond

SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation

Point-Set Anchors for Object Detection, Instance Segmentation and Pose Estimation

Contact Info

Product

Resources

About