Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation

Tang, Chufeng; Chen, Hang; Xiao, Li; Li, Jianmin; Zhang, Zhaoxiang; Hu, Xiaolin

doi:10.1109/cvpr46437.2021.01371

Cited by 68 publications

(45 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mask R-CNN [15], as the representative method, extends Faster R-CNN by adding a mask branch to predict masks for objects and offers a strong baseline for end-to-end instance segmentation. [10,20,35,45] address the low-quality segmentation and coarse boundaries arising in Mask R-CNN and present several approaches to refine the mask predictions for high-quality masks. [3,6] exploit cascade structures to progressively improve the object localization for more accurate mask prediction.…”

Section: Related Workmentioning

confidence: 99%

Sparse Instance Activation for Real-Time Instance Segmentation

Cheng¹,

Wang²,

Chen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a conceptually novel, efficient, and fully convolutional framework for real-time instance segmentation. Previously, most instance segmentation methods heavily rely on object detection and perform mask prediction based on bounding boxes or dense centers. In contrast, we propose a sparse set of instance activation maps, as a new object representation, to highlight informative regions for each foreground object. Then instance-level features are obtained by aggregating features according to the highlighted regions for recognition and segmentation. Moreover, based on bipartite matching, the instance activation maps can predict objects in a one-to-one style, thus avoiding non-maximum suppression (NMS) in post-processing. Owing to the simple yet effective designs with instance activation maps, SparseInst has extremely fast inference speed and achieves 40 FPS and 37.9 AP on the COCO benchmark, which significantly outperforms the counterparts in terms of speed and accuracy. Code and models are available at https://github. com/hustvl/SparseInst.

show abstract

Section: Related Workmentioning

confidence: 99%

Sparse Instance Activation for Real-Time Instance Segmentation

Cheng¹,

Wang²,

Chen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Boundary refinement To tackle the aforementioned limitations, various boundary refinement methods [8,18,20,33,41,42] are proposed. Meanwhile, to evaluate the boundary performance more accurate, a novel evaluation metric Boundary AP [7] is proposed, which can solve the desensitization problem of the current evaluation metric (AP) for object boundary region.…”

Section: Related Workmentioning

confidence: 99%

“…To address this issue, the refinement of boundaries is raised as a new topic in recent years. A novel evaluation metric called Boundary AP [7] and a number of approaches [8,18,33,42] are proposed. Boundary AP [7] focuses on the accuracy of the object boundary region thus can better reflect boundary refinement.…”

Section: Introductionmentioning

confidence: 99%

“…However, existing top-performing refinement methods mainly focus on accuracy and lacks consideration of other two factors which are actually important in the practical application scenarios. For example, Boundary Patch Refinement method [33] proposed to additionally process patches along boundaries after the instance segmentation model, resulting in massive computational cost. RefineMask [42] designed a new architecture built upon Mask-RCNN [13] to refine the quality of instance mask with fine-grained features.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SharpContour: A Contour-based Boundary Refinement Approach for Efficient and Accurate Instance Segmentation

Zhu¹,

Xuanye²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

Excellent performance has been achieved on instance segmentation but the quality on the boundary area remains unsatisfactory, which leads to a rising attention on boundary refinement. For practical use, an ideal post-processing refinement scheme are required to be accurate, generic and efficient. However, most of existing approaches propose pixel-wise refinement, which either introduce a massive computation cost or design specifically for different backbone models. Contour-based models are efficient and generic to be incorporated with any existing segmentation methods, but they often generate over-smoothed contour and tend to fail on corner areas. In this paper, we propose an efficient contour-based boundary refinement approach, named SharpContour, to tackle the segmentation of boundary area. We design a novel contour evolution process together with an Instance-aware Point Classifier. Our method deforms the contour iteratively by updating offsets in a discrete manner. Differing from existing contour evolution methods, SharpContour estimates each offset more independently so that it predicts much sharper and accurate contours. Notably, our method is generic to seamlessly work with diverse existing models with a small computational cost. Experiments show that SharpContour achieves competitive gains whilst preserving high efficiency.

show abstract

“…To be specific, most methods adopt a two-stage object detection architecture [5] for this task by learning an additional mask head to segment objects within box proposals [6][7][8][9]. Recent works focus on high-quality mask segmentation by increasing the prediction resolutions using dynamic networks [10,11] or boundary refinement [12][13][14]. Despite their success, these works all require costly mask annotations of every class.…”

Section: Introductionmentioning

confidence: 99%

Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling

Huynh¹,

Kuen²,

Lin³

et al. 2021

Preprint

View full text Add to dashboard Cite

Open-vocabulary instance segmentation aims at segmenting novel classes without mask annotations. It is an important step toward reducing laborious human supervision. Most existing works first pretrain a model on captioned images covering many novel classes and then finetune it on limited base classes with mask annotations. However, the high-level textual information learned from caption pretraining alone cannot effectively encode the details required for pixel-wise segmentation. To address this, we propose a cross-modal pseudo-labeling framework, which generates training pseudo masks by aligning word semantics in captions with visual features of object masks in images. Thus, our framework is capable of labeling novel classes in captions via their word semantics to self-train a student model. To account for noises in pseudo masks, we design a robust student model that selectively distills mask knowledge by estimating the mask noise levels, hence mitigating the adverse impact of noisy pseudo masks. By extensive experiments, we show the effectiveness of our framework, where we significantly improve mAP score by 4.5% on MS-COCO and 5.1% on the large-scale Open Images & Conceptual Captions datasets compared to the state-of-the-art.

show abstract

Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation

Cited by 68 publications

References 31 publications

Sparse Instance Activation for Real-Time Instance Segmentation

Sparse Instance Activation for Real-Time Instance Segmentation

SharpContour: A Contour-based Boundary Refinement Approach for Efficient and Accurate Instance Segmentation

Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling

Contact Info

Product

Resources

About