Semantic Aware Attention Based Deep Object Co-segmentation

Chen, Hong; Huang, Yifei; Nakayama, Hideki

doi:10.1007/978-3-030-20870-7_27

Cited by 45 publications

(66 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Existing methods in both groups use hand-crafted features such as SIFT [47], HOG [48], or texton [52] for describing a set of object candidates extracted from super-pixels or region-based proposals. Recently, learning based methods [16], [17], [21], [53] have been developed for object co-segmentation. While significant improvement has been shown, these methods [16], [17], [53] require costly foreground masks for training and are not applicable to unseen object categories.…”

Section: Related Workmentioning

confidence: 99%

Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-Segmentation

Chen

Lin

Yang

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

We present an approach for jointly matching and segmenting object instances of the same category within a collection of images. In contrast to existing algorithms that tackle the tasks of semantic matching and object co-segmentation in isolation, our method exploits the complementary nature of the two tasks. The key insights of our method are two-fold. First, the estimated dense correspondence field from semantic matching provides supervision for object co-segmentation by enforcing consistency between the predicted masks from a pair of images. Second, the predicted object masks from object co-segmentation in turn allow us to reduce the adverse effects due to background clutters for improving semantic matching. Our model is end-to-end trainable and does not require supervision from manually annotated correspondences and object masks. We validate the efficacy of our approach on four benchmark datasets: TSS, Internet, PF-PASCAL, and PF-WILLOW, and show that our algorithm performs favorably against the state-of-the-art methods on both semantic matching and object co-segmentation tasks.Index Terms-Semantic matching, object co-segmentation, weakly-supervised learning. !Our work. In this paper, we propose to jointly tackle both semantic matching and object co-segmentation with a twostream network in an end-to-end trainable fashion. Our key insights are two-fold. First, to suppress the effect of background clutters, the predicted object masks by object co-segmentation allow the model to focus on matching the segmented foreground regions while excluding background matching. Second, the estimated dense correspondence fields by semantic matching provide supervision for enforcing the model to generate geometrically consistent object masks across images. Therefore, we exploit the interdependency between the two network outputs, i.e., the estimated dense correspondence fields and the predicted foreground object masks, by introducing the cross-network consistency loss. Incorporating this loss improves both tasks since it encourages arXiv:1906.05857v1 [cs.CV]

show abstract

Section: Related Workmentioning

confidence: 99%

Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-Segmentation

Chen

Lin

Yang

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…[4,32] tackled IOCS through a pair-wise comparison protocol and employed a Siamese network to capture the similarity between two related images. Our AGNN based ICOS solution is significantly different from [4,32]. First, [4,32] consider IOCS as a pair-wise image matching problem, while we formulate IOCS as an information propagation and fusion process among multiple images.…”

Section: Image Object Co-segmentationmentioning

confidence: 99%

“…• PASCAL VOC [11] mance on a subset of Internet (100 images per class are sampled) with mean J . Implementation Details: Following [4,32], we employ PASCAL VOC to train our model. In each iteration, we randomly sample a group of N = 3 images that belong to the same semantic class, and feed two groups with randomly selected classes (6 images in total) to the network.…”

Section: Datasets and Metricsmentioning

confidence: 99%

See 1 more Smart Citation

Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks

Yang

Shen

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

264

133

View full text Add to dashboard Cite

This work proposes a novel attentive graph neural network (AGNN) for zero-shot video object segmentation (ZVOS). The suggested AGNN recasts this task as a process of iterative information fusion over video graphs. Specifically, AGNN builds a fully connected graph to efficiently represent frames as nodes, and relations between arbitrary frame pairs as edges. The underlying pair-wise relations are described by a differentiable attention mechanism. Through parametric message passing, AGNN is able to efficiently capture and mine much richer and higher-order relations between video frames, thus enabling a more complete understanding of video content and more accurate foreground estimation. Experimental results on three video segmentation datasets show that AGNN sets a new state-ofthe-art in each case. To further demonstrate the generalizability of our framework, we extend AGNN to an additional task: image object co-segmentation (IOCS). We perform experiments on two famous IOCS datasets and observe again the superiority of our AGNN model. The extensive experiments verify that AGNN is able to learn the underlying semantic/appearance relationships among video frames or related images, and discover the common objects.

show abstract

“…The co-segmentation model is adopted from Ref. 19 and consists of a Siamese encoder-decoder framework coupled with an attention module. The model takes in a pair of images as input and produces segmentations for each lesion, which are then passed to a densely-connected conditional random field 20 (DCRF) to obtain the final lesion masks.…”

Section: Lesion Co-segmentationmentioning

confidence: 99%

Weakly-supervised lesion segmentation on CT scans using co-segmentation

Agarwal

Tang

Xiao³

et al. 2020

Medical Imaging 2020: Computer-Aided Diagnosis

View full text Add to dashboard Cite

Lesion segmentation on computed tomography (CT) scans is an important step for precisely monitoring changes in lesion/tumor growth. This task, however, is very challenging since manual segmentation is prohibitively timeconsuming, expensive, and requires professional knowledge. Current practices rely on an imprecise substitute called response evaluation criteria in solid tumors (RECIST). Although these markers lack detailed information about the lesion regions, they are commonly found in hospitals' picture archiving and communication systems (PACS). Thus, these markers have the potential to serve as a powerful source of weak-supervision for 2D lesion segmentation. To approach this problem, this paper proposes a convolutional neural network (CNN) based weakly-supervised lesion segmentation method, which first generates the initial lesion masks from the RECIST measurements and then utilizes co-segmentation to leverage lesion similarities and refine the initial masks. In this work, an attention-based co-segmentation model is adopted due to its ability to learn more discriminative features from a pair of images. Experimental results on the NIH DeepLesion dataset demonstrate that the proposed co-segmentation approach significantly improves lesion segmentation performance, e.g the Dice score increases about 4.0% (from 85.8% to 89.8%).

show abstract

Semantic Aware Attention Based Deep Object Co-segmentation

Cited by 45 publications

References 44 publications

Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-Segmentation

Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-Segmentation

Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks

Weakly-supervised lesion segmentation on CT scans using co-segmentation

Contact Info

Product

Resources

About