Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks

Abstract

We present Convolutional Oriented Boundaries (COB), which produces multiscale oriented contours and region hierarchies starting from generic image classification Convolutional Neural Networks (CNNs). COB is computationally efficient, because it requires a single CNN forward pass for multi-scale contour detection and it uses a novel sparse boundary representation for hierarchical segmentation; it gives a significant leap in performance over the state-of-the-art, and it generalizes very well to unseen categories and datasets. Particularly, we show that learning to estimate not only contour strength but also orientation provides more accurate results. We perform extensive experiments for low-level applications on BSDS, PASCAL Context, PASCAL Segmentation, and NYUD to evaluate boundary detection performance, showing that COB provides state-of-the-art contours and region hierarchies in all datasets. We also evaluate COB on high-level tasks when coupled with multiple pipelines for object proposals, semantic contours, semantic segmentation, and object detection on MS-COCO, SBD, and PASCAL; showing that COB also improves the results for all tasks.

Attentive Single-Tasking of Multiple Tasks

“…Edge Detection: For edge detection, we use w t = 50 and binary cross-entropy loss. As is common practice [63,27,36], the positive pixels are weighted more (0.95) than the negative ones (0.05), to account for the class imbalance. When training for a single task in BSDS ( Table 2 of main paper), where there are more than a single annotators, we use the multi-instance learning (MIL) loss of [27].…”

Evaluation of Hierarchical Watersheds

“…As reference state-of-the-art results, we include Multiscale Combinatorial Grouping (MCG) hierarchies from [10], Convolutional Object Boundaries (COB) hierarchies from [55], [58], and Least Effort Segmentation (LEP) from [59] in our assessments. MCG also uses SED as the main cue for contour detection, but then merges several hierarchies (referred to as OWT-UCM in the literature [7]) computed at different scales.…”

“…In our experiments, the test set is composed of the last 2 498 images of the Pascal VOC'10 validation set as proposed in [55]. The object detection measure is evaluated on the MS-COCO [32] and Pascal VOC'12 segmentation [33] datasets.…”

Learning Regional Attraction for Line Segment Detection

“…It then inspires researchers to upgrade the existing edge-based line segment detectors to deep-edge based line segment detectors. Convolutional Oriented Boundaries (COB) [23], [42] detector was proposed to get multi-scale oriented contours and region hierarchies from a single ConvNet. Since the oriented contours are adaptive to the input format (i.e., edge pixels and orientations) of fast LSD [2], they can be used to address the issue of incomplete detection in LSD effectively.…”