Half-integrality based algorithms for cosegmentation of images

Abstract

We study the cosegmentation problem where the objective is to segment the same object (i.e., region) from a pair of images. The segmentation for each image can be cast using a partitioning/segmentation function with an additional constraint that seeks to make the histograms of the segmented regions (based on intensity and texture features) similar. Using Markov Random Field (MRF) energy terms for the simultaneous segmentation of the images together with histogram consistency requirements using the squared L 2 (rather than L 1) distance, after linearization and adjustments, yields an optimization model with some interesting combinatorial properties. We discuss these properties which are closely related to certain relaxation strategies recently introduced in computer vision. Finally, we show experimental results of the proposed approach.

Cosegmentation Revisited: Models and Optimization

“…We also consider a new model, which is a straightforward extension from a single to multiple images of Boykov-Jolly [4]. For a fair comparison we improved on existing optimization methods for the models in [1,2]. We achieved this by using a Dual Decomposition technique.…”

“…To achieve this we limit ourselves to the task of cosegmenting two images only, with color as the only appearance feature, and where distributions are expressed in terms of histograms. We consider three existing models [1][2][3], which differ only in the distance measure between the two color histograms. We also consider a new model, which is a straightforward extension from a single to multiple images of Boykov-Jolly [4].…”

“…Let us be more precise on the definition of the "same" foreground object. In this paper we use the definition of [1][2][3] where the only constraint is that the distribution of some appearance features of the foreground region in each image have to be similar. The appearance features can encode different information, like color and texture, and various similarity measures can be envisioned.…”

“…We also study a new model, which is a straightforward extension of the Boykov-Jolly model for single image segmentation [4]. In terms of optimization, we use a Dual Decomposition (DD) technique in addition to optimization methods in [1,2]. Experiments show a significant improvement of DD over published methods.…”

“…However, no comparison of such models and corresponding optimization techniques has been done so far. We analyze three existing models: the L1 norm model of Rother et al [1], the L2 norm model of Mukherjee et al [2] and the "reward" model of Hochbaum and Singh [3]. We also study a new model, which is a straightforward extension of the Boykov-Jolly model for single image segmentation [4].…”