Loosecut: Interactive image segmentation with loosely bounded boxes

Abstract

One popular approach to interactively segment the foreground object of interest from an image is to annotate a bounding box that covers the foreground object. Then, a binary labeling is performed to achieve a refined segmentation. One major issue of the existing algorithms for such interactive image segmentation is their preference of an input bounding box that tightly encloses the foreground object. This increases the annotation burden, and prevents these algorithms from utilizing automatically detected bounding boxes. In this paper, we develop a new LooseCut algorithm that can handle cases where the input bounding box only loosely covers the foreground object. We propose a new Markov Random Fields (MRF) model for segmentation with loosely bounded boxes, including a global similarity constraint to better distinguish the foreground and background, and an additional energy term to encourage consistent labeling of similar-appearance pixels. This MRF model is then solved by an iterated max-flow algorithm. In the experiments, we evaluate LooseCut in three publicly-available image datasets, and compare its performance against several state-of-the-art interactive image segmentation algorithms. We also show that LooseCut can be used for enhancing the performance of unsupervised video segmentation and image saliency detection.

Dominant Sets for “Constrained” Image Segmentation

“…Table 3 and the pictures in Figure 5 show the respective error rates and the several qualitative segmentation results. Most of the results, CDS-Best Sigma CDS-Self Tuning CDS-Single Sigma Graph-Cut [41] RS [37] Subrs method [40] BNC [54] 3, are reported by previous works [10], [5], [4], [41], [42]. Segmentation Using Loose Bounding Box.…”

“…For the sake of comparison, we conduct the same experiments as in [10]: 41 images out of the 50 GrabCut dataset [3] are selected as the rest 9 images contain multiple objects while the ground truth is only annotated on a single object. As other objects, which are not marked as an object of interest in the ground truth, may be covered when the looseness of the box increases, images of multiple objects are not applicable for testing the loosely bounded boxes [10]. Table 3 summarizes the results of different approaches using bounding box at different level of looseness.…”

Interactive Image Segmentation Using Constrained Dominant Sets

“…The approach not only avoids the annotation burden but also allows the algorithm to use automatically detected bounding boxes which might not tightly encloses the foreground object. It has been shown, in [9], that the well-known GrabCut algorithm [1] fails when the looseness of the box is increased. Our framework, like [9], is able to extract the object of interest in both tight and loose boxes.…”

“…For the sake of comparison, we conduct the same experiments as in [9]: 41 images out of the 50 GrabCut dataset [1] are selected as the rest 9 images contain multiple objects while the ground truth is only annotated on a single object. As other objects, which are not marked as an object of interest in the ground truth, may be covered when the looseness of the box increases, images of multiple objects are not applicable for testing the loosely bounded boxes [9]. Table 3 summarizes the results of different approaches using bounding box at different level of looseness.…”

“…For example, Wu et al [3] claim that bounding boxes are the most natural and economical form in terms of the amount of user interaction, and develop a multiple instance learning algorithm that extracts an arbitrary object located inside a tight bounding box at unknown location. Yu et al [9] also support the bounding-box approach, though their algorithm is different from others in that it does not need bounding boxes tightly enclosing the object of interest, whose production of course increases the annotation burden. They provide an algorithm, based on a Markov Random Field (MRF) energy function, that can handle arXiv:1608.00641v2 [cs.CV] 3 Aug 2016 input bounding box that only loosely covers the foreground object.…”