SCOPS: Self-Supervised Co-Part Segmentation

Abstract

Figure 1: Robustness to variations. Sample part segmentation obtained by SCOPS on different types of image collections: (left) unaligned faces from CelebA [29], (middle) birds from CUB [44] and (right) horses from PASCAL VOC [11] dataset images, showing that SCOPS can be robust to appearance, viewpoint and pose variations. AbstractParts provide a good intermediate representation of objects that is robust with respect to the camera, pose and appearance variations. Existing works on part segmentation is dominated by supervised approaches that rely on large amounts of manual annotations and can not generalize to unseen object categories. We propose a self-supervised deep learning approach for part segmentation, where we devise several loss functions that aids in predicting part segments that are geometrically concentrated, robust to object variations and are also semantically consistent across different object instances. Extensive experiments on different types of image collections demonstrate that our approach can produce part segments that adhere to object boundaries and also more semantically consistent across object instances compared to existing self-supervised techniques.

DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

“…Existing datasets [43,88] are relatively small in size, and only provide sparse correspondence ground truths since manually annotating dense ones is prohibitive. In light of this challenge, weakly supervised semantic correspondence are proposed to learn correspondence without correspondence ground truths [25][26][27][28][29][30]. In addition, existing benchmarks and methods have predominantly focused on "objectcentric" scenarios where each image is occupied by a major object.…”

“…Even though the advantage of learning correspondences and instance segmentation jointly is clear, many state of the art methods do not make use of this approach due to the lack of large scale datasets with both masks and correspondences. To overcome this challenge, weakly supervised methods have been recently introduced to relax the need for costly supervision in both tasks [25][26][27][28][29][30][46][47][48][49]. Our work is aligned with these efforts as we aim to address instance segmentation and semantic correspondence jointly with inexpensive bounding box supervision.…”

Self-Supervision By Prediction For Object Discovery In Videos

“…Mask Concentration Loss: In order to promote compactness for object masks, we use a geometric concentration loss as in [35]. The tuple (x, y) in Eq.…”

MixSiam: A Mixture-based Approach to Self-supervised Representation Learning

“…Learning discriminative image representation in an unsupervised/ self-supervised manner has attracted increasing interest (Agrawal, Carreira, and Malik 2015;Doersch, Gupta, and Efros 2015;Xie et al 2021), for it gets rid of the costly manually-labeled data and achieves promising performance on many down-stream tasks (Larsson et al 2019;Hung et al 2019;Doersch and Zisserman 2017). These methods generally design pretext tasks and learn the representation from the label generated by the tasks, such as rotation predicting (Komodakis and Gidaris 2018), jigsaw (Noroozi and Favaro 2016;Kim et al 2018), in-painting (Pathak et al 2016), colorization (Zhang, Isola, and Efros 2016;Larsson, Maire, and Shakhnarovich 2017) and clustering (Noroozi et al 2018;Caron et al 2018).…”