Affinity Derivation and Graph Merge for Instance Segmentation

Liu, Yiding; Yang, Siyu; Li, Bin; Zhou, Wengang; Xu, Jizheng; Li, Houqiang; Lu, Yan

doi:10.1007/978-3-030-01219-9_42

Cited by 102 publications

(105 citation statements)

References 53 publications

Supporting

Mentioning

105

Contrasting

Order By: Relevance

“…The concept of learning pixel-pair affinity has been developed in many previous works [36,23,1,4,42] to facilitate semantic segmentations during training or postprocessing. Recently, Liu et al [40] propose learning instance-aware affinity and grouping pixels into instances with agglomerative hierarchical clustering. Our approach also utilizes instance-aware affinity to distinguish object instances, but both the ways to derive affinities and group pixels are significantly different.…”

Section: Pixel-pair Affinitymentioning

confidence: 99%

“…Our approach also utilizes instance-aware affinity to distinguish object instances, but both the ways to derive affinities and group pixels are significantly different. Importantly, Liu et al [40] employ two models and require multiple passes for the RoIs generated from semantic segmentation results. Instead, our approach is single-shot, which requires only one single pass to generate the final instance segmentation result.…”

Section: Pixel-pair Affinitymentioning

confidence: 99%

“…Current state-of-the-art solutions to this challenging problem can be classified into the proposal-based and proposal-free approaches [34,28,40]. The proposal-based approaches regard it as an extension to the classic object detection task [46,39,44,15].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

SSAP: Single-Shot Instance Segmentation With Affinity Pyramid

Gao

Shan

Wang

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

237

146

View full text Add to dashboard Cite

Recently, proposal-free instance segmentation has received increasing attention due to its concise and efficient pipeline. Generally, proposal-free methods generate instance-agnostic semantic segmentation labels and instance-aware features to group pixels into different object instances. However, previous methods mostly employ separate modules for these two sub-tasks and require multiple passes for inference. We argue that treating these two subtasks separately is suboptimal. In fact, employing multiple separate modules significantly reduces the potential for application. The mutual benefits between the two complementary sub-tasks are also unexplored. To this end, this work proposes a single-shot proposal-free instance segmentation method that requires only one single pass for prediction. Our method is based on a pixel-pair affinity pyramid, which computes the probability that two pixels belong to the same instance in a hierarchical manner. The affinity pyramid can also be jointly learned with the semantic class labeling and achieve mutual benefits. Moreover, incorporating with the learned affinity pyramid, a novel cascaded graph partition module is presented to sequentially generate instances from coarse to fine. Unlike previous time-consuming graph partition methods, this module achieves 5× speedup and 9% relative improvement on Average-Precision (AP). Our approach achieves state-of-the-art results on the challenging Cityscapes dataset.

show abstract

Section: Pixel-pair Affinitymentioning

confidence: 99%

Section: Pixel-pair Affinitymentioning

confidence: 99%

See 1 more Smart Citation

SSAP: Single-Shot Instance Segmentation With Affinity Pyramid

Gao

Shan

Wang

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

237

146

View full text Add to dashboard Cite

show abstract

“…They do not assign region of interest for each object instance. Instead, they produce pixelwise predictions of cues such as directional vectors [32], pairwise affinity [34], watershed energy [2], and semantic classes, and then group object instances from the cues in the post-processing stage.…”

Section: Related Workmentioning

confidence: 99%

ShapeMask: Learning to Segment Novel Objects by Refining Shape Priors

Kuo

Angelova

Malik

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

126

View full text Add to dashboard Cite

Instance segmentation aims to detect and segment individual objects in a scene. Most existing methods rely on precise mask annotations of every category. However, it is difficult and costly to segment objects in novel categories because a large number of mask annotations is required. We introduce ShapeMask, which learns the intermediate concept of object shape to address the problem of generalization in instance segmentation to novel categories. Shape-Mask starts with a bounding box detection and gradually refines it by first estimating the shape of the detected object through a collection of shape priors. Next, ShapeMask refines the coarse shape into an instance level mask by learning instance embeddings. The shape priors provide a strong cue for object-like prediction, and the instance embeddings model the instance specific appearance information. Shape-Mask significantly outperforms the state-of-the-art by 6.4 and 3.8 AP when learning across categories, and obtains competitive performance in the fully supervised setting. It is also robust to inaccurate detections, decreased model capacity, and small training data. Moreover, it runs efficiently with 150ms inference time and trains within 11 hours on TPUs. With a larger backbone model, ShapeMask increases the gap with state-of-the-art to 9.4 and 6.2 AP across categories. Code will be released.

show abstract

“…Inspired by recent works [16,22,33] in general semantic instance segmentation, we aim to design a segmentation-based Single-shot Arbitrarily-Shaped Text detector (SAST), which integrates both the high-level object knowledge and low-level pixel information in a single shot and detects scene text of arbitrary shapes with high accuracy and efficiency. Employing a FCN [27] model, various geometric properties of text regions, including text center line (TCL), text border offset (TBO), text center offset (TCO), and text vertex offset (TVO), are designed to learn simultaneously under a multi-task learning formulation.…”

Section: Introductionmentioning

confidence: 99%

A Single-Shot Arbitrarily-Shaped Text Detector based on Context Attended Multi-Task Learning

Wang

Zhang

et al. 2019

Proceedings of the 27th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Detecting scene text of arbitrary shapes has been a challenging task over the past years. In this paper, we propose a novel segmentationbased text detector, namely SAST, which employs a context attended multi-task learning framework based on a Fully Convolutional Network (FCN) to learn various geometric properties for the reconstruction of polygonal representation of text regions. Taking sequential characteristics of text into consideration, a Context Attention Block is introduced to capture long-range dependencies of pixel information to obtain a more reliable segmentation. In post-processing, a Point-to-Quad assignment method is proposed to cluster pixels into text instances by integrating both high-level object knowledge and low-level pixel information in a single shot. Moreover, the polygonal representation of arbitrarily-shaped text can be extracted with the proposed geometric properties much more effectively. Experiments on several benchmarks, including ICDAR2015, ICDAR2017-MLT, SCUT-CTW1500, and Total-Text, demonstrate that SAST achieves better or comparable performance in terms of accuracy. Furthermore, the proposed algorithm runs at 27.63 FPS on SCUT-CTW1500 with a Hmean of 81.0% on a single NVIDIA Titan Xp graphics card, surpassing most of the existing segmentation-based methods.

show abstract

Affinity Derivation and Graph Merge for Instance Segmentation

Cited by 102 publications

References 53 publications

SSAP: Single-Shot Instance Segmentation With Affinity Pyramid

SSAP: Single-Shot Instance Segmentation With Affinity Pyramid

ShapeMask: Learning to Segment Novel Objects by Refining Shape Priors

A Single-Shot Arbitrarily-Shaped Text Detector based on Context Attended Multi-Task Learning

Contact Info

Product

Resources

About