Curvature regularity for region-based image segmentation and inpainting: A linear programming relaxation

Abstract

We consider a class of region-based energies for image segmentation and inpainting which combine region integrals with curvature regularity of the region boundary. To minimize such energies, we formulate an integer linear program which jointly estimates regions and their boundaries. Curvature regularity is imposed by respective costs on pairs of adjacent boundary segments.By solving the associated linear programming relaxation and thresholding the solution one obtains an approximate solution to the original integer problem. To our knowledge this is the first approach to impose curvature regularity in region-based formulations in a manner that is independent of initialization and allows to compute a bound on the optimal energy.In a variety of experiments on segmentation and inpainting, we demonstrate the advantages of higher-order regularity. Moreover, we demonstrate that for most experiments the optimality gap is smaller than 2% of the global optimum. For many instances we are even able to compute the global optimum.

Curvature Regularity for Multi-label Problems - Standard and Customized Linear Programming

“…In this section, we review the formulation in [25] for region-based binary segmentation problems with curvature regularity (but differ slightly in notation).…”

“…For inpainting, we show that the parameterization of [25] may degenerate (and does so quite often in practice) since "level lines" are allowed to cross between different levels. We discuss what is needed to prevent this and then introduce a new multi-label parameterization to remedy the situation that is still practicable (in terms of memory and run-time).…”

“…In a recent work, Schoenemann et al [25,26] proposed to solve region-based problems with curvature regularity as linear programs. This approach is independent of initialization and has the advantage over the boundary-based methods that region-terms can be included, self-intersections be excluded and that the computed regions need not be (singly-) connected.…”

“…For segmentation we derive a "curvature Potts model", i.e. show that the linear program in [25] can be generalized to more than two regions. Here, we make use of prior work [14,32] on writing the (length-based) Potts model as a very compact integer linear program.…”

“…We follow recent work by Schoenemann et al [25] for expressing curvature regularity as a linear program. While the original formulation focused on binary segmentation, we address several multi-label problems, including segmentation, denoising and inpainting, all cast as a single linear program.…”