Convexity Shape Constraints for Image Segmentation

Abstract

Segmenting an image into multiple components is a central task in computer vision. In many practical scenarios, prior knowledge about plausible components is available. Incorporating such prior knowledge into models and algorithms for image segmentation is highly desirable, yet can be non-trivial. In this work, we introduce a new approach that allows, for the first time, to constrain some or all components of a segmentation to have convex shapes. Specifically, we extend the Minimum Cost Multicut Problem by a class of constraints that enforce convexity. To solve instances of this APX-hard integer linear program to optimality, we separate the proposed constraints in the branch-and-cut loop of a state-of-the-art ILP solver. Results on natural and biological images demonstrate the effectiveness of the approach as well as its advantage over the state-of-the-art heuristic.

A Level Set Representation Method for $N$-Dimensional Convex Shape and Applications

“…Among them, the convexity prior is one of the most important priors. Firstly, many objects in natural and biomedical images are convex, such as balls, buildings, and some organs [30]. Secondly, convexity also plays a very important role in many computer vision tasks, like human vision completion [23].…”

“…This method was then generalized for multiple convex objects segmentation in [13]. In [30], the authors proposed a segmentation model that can handle multiple convex objects. They formulated the problem as a minimum cost multi-cut problem with a novel convexity constraint: If a path is inside the concerned object, then the line segment between the two ends should not pass through the object boundary.…”

“…For the discrete methods in [15,30], it is almost impossible to extend these methods to higher dimensional (≥ 3) cases directly because of the computational complexity. One can set up the same model as the two-dimensional case, but the computational cost will increase dramatically.…”

Injecting spatial priors in Earth observation with machine vision

“…(Bai and Urtasun, 2017) refine an existing semantic segmentation map by predicting a distance transform to the nearest boundary. High level relationships are accounted for in (Royer et al, 2016;Zhang et al, 2016b) by means of an instance MRF applied to the CNN's output.…”

“…values weighting the snakes' energy function terms on a per-pixel basis, and learns them using a CNN. Although this work focuses on curvature priors useful for segmenting objects of polygonal shape, other priors can be enforced with ACMs, such as convexity for biomedical imaging (Royer et al, 2016).…”