Robust higher order potentials for enforcing label consistency

Abstract

This paper proposes a novel framework for labelling problems which is able to combine multiple segmentations in a principled manner. Our method is based on higher order conditional random fields and uses potentials defined on sets of pixels (image segments) generated using unsupervised segmentation algorithms. These potentials enforce label consistency in image regions and can be seen as a strict generalization of the commonly used pairwise contrast sensitive smoothness potentials. The higher order potential functions used in our framework take the form of the Robust P n model. This enables the use of powerful graph cut based move making algorithms for performing inference in the framework [14]. We test our method on the problem of multi-class object segmentation by augmenting the conventional CRF used for object segmentation with higher order potentials defined on image regions. Experiments on challenging data sets show that integration of higher order potentials quantitatively and qualitatively improves results leading to much better definition of object boundaries. We believe that this method can be used to yield similar improvements for many other labelling problems.

Discriminatively Trained Dense Surface Normal Estimation

“…The predictions using context based approaches are typically very noisy, do not follow object boundaries, and thus require some form of regularization [28,27]. The rectangle-based context representation is typically more robust and leads to better performance quantitatively [24,25,27].…”

“…The notion of their quality largely depends on the task they are applied to. For semantic segmentation similar sized segments have typically more discriminant feature representations, but methods producing segments of different scales are more suitable for enforcing label consistency in segments [28]. For normal estimation a single unsupervised segmentation method can not produce label-consistent segments in general, e.g.…”

“…In [32], the feature representation of segments also includes a feature representation of the union of neighbours to encode contextual information. Alternatively in [27] multiple segmentations are combined in the CRF framework by finding the smooth labelling that agrees with most of the predictions of individual pixel-wise and segment-wise classifiers and enforces label-consistency of segments as a soft constraint (see also [28]).…”

MaxFlow Revisited: An Empirical Comparison of Maxflow Algorithms for Dense Vision Problems

“…The central thesis of this work is that since this comparison a decade ago, the models used in computer vision and the kinds of inference problems we solve have changed significantly. Specifically, while [5] only considered 4-connected grid MRFs, the models today involve high-order terms [26], long-range connections [31], hierarchical MRFs [28] and even global terms [30]. The effect of all these modifications is to make the underlying max-flow graph significantly denser, thus causing the complexity of the algorithm of [5] to become a concern.…”

“…Max-flow also plays a crucial role in approximate minimization of energy functions with multi-label variables [4,6], triplet or higher order terms [26,27,35,37], global terms [30], and terms encoding label costs [11,32].…”