Finding Things: Image Parsing with Regions and Per-Exemplar Detectors

Abstract

This paper presents a system for image parsing, or labeling each pixel in an image with its semantic category, aimed at achieving broad coverage across hundreds of object categories, many of them sparsely sampled. The system combines region-level features with per-exemplar sliding window detectors. Per-exemplar detectors are better suited for our parsing task than traditional bounding box detectors: they perform well on classes with little training data and high intra-class variation, and they allow object masks to be transferred into the test image for pixel-level segmentation. The proposed system achieves state-of-theart accuracy on three challenging datasets, the largest of which contains 45,676 images and 232 labels.

CollageParsing: Nonparametric Scene Parsing by Adaptive Overlapping Windows

“…Compared with Tighe and Lazebnik's extended SuperParsing algorithm with per-exemplar detectors and a combination SVM [33], CollageParsing obtains 1.9% higher per-class accuracy and 1.5% lower per-pixel accuracy. Compared with Farabet et al's system [9] with "natural" training, at a tradeoff of 1.4% lower per-pixel accuracy an 11.5% per-class accuracy improvement is obtained.…”

“…As discussed in Section 1, this characteristic makes CollageParsing (and other nonparametric approaches) particularly well suited for open-universe datasets since no model retraining is required as the dataset expands. On a dataset of 45,000 images with 232 semantic labels, just the per-exemplar detector component of Tighe and Lazebnik [33] requires four days on a 512-node cluster to train. On a dataset of 715 images with eight semantic labels [13], Farabet et al [9] requires "48h on a regular server" to train.…”

“…State-of-the-art parametric Farabet et al [9], "natural" 78.5 29.6 Farabet et al [9], "balanced" 74.2 46.0 Tighe and Lazebnik [33] 78.6 39.2…”

“…Tighe and Lazebnik [33] recently extended the SuperParsing algorithm with per-exemplar detectors (Exemplar-SVMs [23]). The data term based on superpixel matching is the same as in the SuperParsing algorithm.…”

“…Parametric scene parsing methods have a small advantage in accuracy over nonparametric methods, however as a tradeoff they require large amounts of model training (for example, training just the per-exemplar detector component of the extended SuperParsing algorithm [33] on a dataset of 45,000 images requires four days on a 512-node cluster), making them less practical for openuniverse datasets. As we show in the experiments, CollageParsing achieves stateof-the-art results among nonparametric scene parsing methods, and comparable performance with state-of-the-art parametric methods while not requiring expensive model training.…”