Contextually guided semantic labeling and search for three-dimensional point clouds

Abstract

RGB-D cameras, which give an RGB image together with depths, are becoming increasingly popular for robotic perception. In this paper, we address the task of detecting commonly found objects in the 3D point cloud of indoor scenes obtained from such cameras. Our method uses a graphical model that captures various features and contextual relations, including the local visual appearance and shape cues, object co-occurence relationships and geometric relationships. With a large number of object classes and relations, the model's parsimony becomes important and we address that by using multiple types of edge potentials. We train the model using a maximum-margin learning approach. In our experiments over a total of 52 3D scenes of homes and offices (composed from about 550 views), we get a performance of 84.06% and 73.38% in labeling office and home scenes respectively for 17 object classes each. We also present a method for a robot to search for an object using the learned model and the contextual information available from the current labelings of the scene. We applied this algorithm successfully on a mobile robot for the task of finding 12 object classes in 10 different offices and achieved a precision of 97.56% with 78.43% recall. 1

Querying 3D Data by Adjacency Graphs

“…A popular approach to model semantic properties of a space has been to study graphs constructed over-segmented scenes [7,8]. Our approach is to similarly construct an adjacency graph over the scene but to instead identify topological structures within that graph.…”

“…Those often employ graphical models over adjacent areas to infer semantic labels, primarily by using some kind of probabilistic inference over the graph. An early example of this kind of inference on a stitched point cloud map was presented by Anand et al [7]. As is natural in a 3D context, they use e.g.…”

Automated Classification of Heritage Buildings for as-Built Bim Using Machine Learning Techniques

“…By connecting several nodes into a graph, probabilistic reasoning allows the likelihood maximisation of the labels of the nodes. Markov Random Fields (MRF) and Conditional Random Fields (CRF) were proposed for the classification of indoor scenes (Koppula et al, 2013, Anand et al, 2012, Gerke and Xiao, 2014, Niemeyer et al, 2014. Similar approaches were utilized for close range terrestrial classification (Lim and Suter, 2009).…”

“…In general, this Scan-to-BIM process consists of the following steps. First, the points are clustered into groups using statistical procedures (Anand et al, 2012). The grouped points are replaced by primitives for computational efficiency.…”

Physically Grounded Spatio-temporal Object Affordances

“…Recent works in computer vision have revisited these aspects. For example, the symbolic notion of affordances can be interpreted as an object attribute labeling problem [25,36,7,3,42], and more Fig. 1.…”