What, Where and How Many? Combining Object Detectors and CRFs

Abstract

Computer vision algorithms for individual tasks such as object recognition, detection and segmentation have shown impressive results in the recent past. The next challenge is to integrate all these algorithms and address the problem of scene understanding. This paper is a step towards this goal. We present a probabilistic framework for reasoning about regions, objects, and their attributes such as object class, location, and spatial extent. Our model is a Conditional Random Field defined on pixels, segments and objects. We define a global energy function for the model, which combines results from sliding window detectors, and low-level pixel-based unary and pairwise relations. One of our primary contributions is to show that this energy function can be solved efficiently. Experimental results show that our model achieves significant improvement over the baseline methods on CamVid and PASCAL VOC datasets.

Semantic Video Segmentation from Occlusion Relations within a Convex Optimization Framework

“…However, we train and test on only the 11 classes shown in Fig. 2 as common practice when evaluating on CamVid [7,11,23]. For our framework, we consider cars, sign-symbol, pedestrian, column-pole, and bicyclist as the object classes and building, tree, sky, road, fence, and sidewalk as the background classes.…”

“…vertical, horizontal, sky) [8], and a model for camera viewpoint [9], which can provide powerful constraints on the plausible locations of objects in the scene [10]. [11] takes advantage of powerful sliding window detectors [12,13] to facilitate the labeling of objects (e.g. cars, persons) as they project to smaller regions in the image than background classes (e.g.…”

“…building, road, or sky), an issue our approach also addresses. Incorporating multiple cues facilitates [5,11,8,9] the joint optimization of coupled problems (e.g. semantic labels and depth estimates).…”

Sensor fusion for semantic segmentation of urban scenes

“…Kohli et al [15] formulate Robust P N clique potentials. The technique was used by Ladicky et al [16] to incorporate sliding window object detections, and extended into a multilevel hierarchical manner by Ladicky et al [17]. Using clique potentials comes at a cost of run-time, and solving for optimal parameters remains an open problem.…”

Extraction of Objects from Terrestrial Laser Scans by Integrating Geometry Image and Intensity Data with Demonstration on Trees

“…Recent works, mainly related to scene understanding from images (e.g., [12][13][14]) have demonstrated how the application of segmentation processes for recognition tasks yields promising results, both in terms of object class recognition and correcting the segmentation of the searched objects. Such approaches are categorized as joint categorization and segmentation (JCaS).…”