Geometry Driven Semantic Labeling of Indoor Scenes

Khan, Salman; Bennamoun, Mohammed; Sohel, Ferdous; Togneri, Roberto

doi:10.1007/978-3-319-10590-1_44

Cited by 51 publications

(45 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, they do not go beyond the depth-based features or priors. In this paper, we show how to incorporate depth information into the various components of a random field model and then evaluate the contribution made by each component in enhancing semantic labeling performance (Khan et al 2014b). Our framework is particularly inspired by the works on semantic labeling of RGBD data (Silberman and Fergus 2011;Silberman et al 2012), considering long-range interactions ), parametric learning (Szummer et al 2008;Tsochantaridis et al 2004) and geometric reconstruction (Rabbani et al 2006).…”

Section: Related Workmentioning

confidence: 99%

Integrating Geometrical Context for Semantic Labeling of Indoor Scenes using RGBD Images

et al. 2015

Self Cite

View full text Add to dashboard Cite

Inexpensive structured light sensors can capture rich information from indoor scenes, and scene labeling problems provide a compelling opportunity to make use of this information. In this paper we present a novel conditional random field (CRF) model to effectively utilize depth information for semantic labeling of indoor scenes. At the core of the model, we propose a novel and efficient plane detection algorithm which is robust to erroneous depth maps. Our CRF formulation defines local, pairwise and higher order interactions between image pixels. At the local level, we propose a novel scheme to combine energies derived from appearance, depth and geometry-based cues. The proposed local energy also encodes the location of each object class by considering the approximate geometry of a scene. For the pairwise interactions, we learn a boundary measure which defines the Communicated by spatial discontinuity of object classes across an image. To model higher-order interactions, the proposed energy treats smooth surfaces as cliques and encourages all the pixels on a surface to take the same label. We show that the proposed higher-order energies can be decomposed into pairwise submodular energies and efficient inference can be made using the graph-cuts algorithm. We follow a systematic approach which uses structured learning to fine-tune the model parameters. We rigorously test our approach on SUN3D and both versions of the NYU-Depth database. Experimental results show that our work achieves superior performance to stateof-the-art scene labeling techniques.

show abstract

Section: Related Workmentioning

confidence: 99%

Integrating Geometrical Context for Semantic Labeling of Indoor Scenes using RGBD Images

et al. 2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…Couprie et al [29] 64.5 63.5 Khan et al [30] 69.2 65.6 Stückler et al [25] 70.9 67.0 Müller and Behnke [19] 72.3 71.9 Wolf et al [18] 72. networks, the latter model required training in two steps. Additionally, the feature learning at multiple scales approach in [5] seems to be beneficial, when compared to our single scale model.…”

Section: Accuracy (%)mentioning

confidence: 99%

Combining Semantic and Geometric Features for Object Class Segmentation of Indoor Scenes

Husain

Schulz

Dellen

et al. 2017

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Abstract-Scene understanding is a necessary prerequisite for robots acting autonomously in complex environments. Low-cost RGB-D cameras such as Microsoft Kinect enabled new methods for analyzing indoor scenes and are now ubiquitously used in indoor robotics. We investigate strategies for efficient pixelwise object class labeling of indoor scenes that combine both pretrained semantic features transferred from a large color image dataset and geometric features, computed relative to the room structures, including a novel distance-from-wall feature, which encodes the proximity of scene points to a detected major wall of the room. We evaluate our approach on the popular NYU v2 dataset. Several deep learning models are tested, which are designed to exploit different characteristics of the data. This includes feature learning with two different pooling sizes. Our results indicate that combining semantic and geometric features yields significantly improved results for the task of object class segmentation.

show abstract

“…The dataset contains 795 training and 654 testing samples. The training set is used to learn the network parameters as explained in Section III-B and 3 Available at http://cs.nyu.edu/ ∼ silberman/datasets/nyu depth v2.html Couprie et al [29] 64.5 63.5 Khan et al [30] 69.2 65.6 Stückler et al [31] 70.9 67.0 Müller and Behnke [32] 72.3 71.9 Wolf et al [33] 72.6 74.1 Eigen and Fergus [17] 79.1 80.6 Husain et al [4] …”

Section: A Semantic Segmentationmentioning

confidence: 99%

Semantic segmentation priors for object discovery

García

Husain

Schulz

et al. 2016

2016 23rd International Conference on Pattern Recognition (ICPR)

View full text Add to dashboard Cite

Abstract-Reliable object discovery in realistic indoor scenes is a necessity for many computer vision and service robot applications. In these scenes, semantic segmentation methods have made huge advances in recent years. Such methods can provide useful prior information for object discovery by removing false positives and by delineating object boundaries. We propose a novel method that combines bottom-up object discovery and semantic priors for producing generic object candidates in RGB-D images. We use a deep learning method for semantic segmentation to classify colour and depth superpixels into meaningful categories. Separately for each category, we use saliency to estimate the location and scale of objects, and superpixels to find their precise boundaries. Finally, object candidates of all categories are combined and ranked. We evaluate our approach on the NYU Depth V2 dataset and show that we outperform other state-of-the-art object discovery methods in terms of recall.

show abstract

Geometry Driven Semantic Labeling of Indoor Scenes

Cited by 51 publications

References 31 publications

Integrating Geometrical Context for Semantic Labeling of Indoor Scenes using RGBD Images

Integrating Geometrical Context for Semantic Labeling of Indoor Scenes using RGBD Images

Combining Semantic and Geometric Features for Object Class Segmentation of Indoor Scenes

Semantic segmentation priors for object discovery

Contact Info

Product

Resources

About