Multi-Class Segmentation with Relative Location Prior

Abstract

Multi-class image segmentation has made significant advances in recent years through the combination of local and global features. One important type of global feature is that of inter-class spatial relationships. For example, identifying "tree" pixels indicates that pixels above and to the sides are more likely to be "sky" whereas pixels below are more likely to be "grass." Incorporating such global information across the entire image and between all classes is a computational challenge as it is image-dependent, and hence, cannot be precomputed.In this work we propose a method for capturing global information from inter-class spatial relationships and encoding it as a local feature. We employ a two-stage classification process to label all image pixels. First, we generate predictions which are used to compute a local relative location feature from learned relative location maps. In the second stage, we combine this with appearance-based features to provide a final segmentation. We compare our results to recent published results on several multiclass image segmentation databases and show that the incorporation of relative location information allows us to significantly outperform the current state-of-the-art.

Segmentation-based multi-class semantic object detection

“…The fact that the best results are achieved by edison contributes to the idea that an accurate segmentation improves the recognition, as was highlighted by other authors [14,16,18], Yet very different segmentation such as the high over-segmentation given by the grid and the coarser but accurate graphbased segmentation lead to the same recognition accuracy (overall). Nevertheless, in the case of single segmentation recognition, an over-segmentation should be favored since it appears to be more robust for the recognition of the different classes, despite the partial view of an object provided by the segments.…”

“…We split the database in two halves for training and testing. Following the protocol of the other works [26,33,14,36], we ignored the void class during training and evaluation, and horse and mountain classes were also ignored due to insuffiencient amount of representative in the database. The reported results are the pixel-wise accuracy, that is the percentage of pixels correctly classified with respect to the total number of pixels in the segmentation ground truth.…”

“…Note that the results were not obtained using the same training and test datasets. Gould et al [14] used 40% images for training, 60% for test, Shotton et al [26], Verbeek and Triggs [33] split the dataset in two halves, Yang et al [36] reports the average results over a 5-fold cross validation. Our best method (merging multiple segmentation with mean operator) ranks second with for global pixel-wise accuracy with 76.1 behind Gould (76.5).…”

“…Yang et al integrated appearance based classification of meanshift patches, a bag-of-keypoints model and global shape based Elliptical Fourier descriptors within a unified framework for object-based segmentation. Gould et al proposed to combine appearance-based features computed on superpixels patches with relative location priors in a two stage classification process [14]. Malisiewicz and Efros have also shown that using image segmentation was efficient to improve the spatial support for object detection and recognition [18].…”

Structure detection and segmentation of documents using 2D stochastic context-free grammars

“…Within this formal framework, in [17] a solution is proposed for classifying the different textual zones that are present in marriage license books, although no structure detection is performed. In that research, pixel classification based on texture features obtained from the Gabor transform are compared with Relative Location Features [18]. Both sort of features are combined in a Conditional Random Field [19] to take into account contextual information in the classification process of the pixels.…”