Automated Spatial-Semantic Modeling with Applications to Place Labeling and Informed Search

Abstract

This paper presents a spatial-semantic modeling system featuring automated learning of object-place relations from an online annotated database, and the application of these relations to a variety of real-world tasks. The system is able to label novel scenes with place information, as we demonstrate on test scenes drawn from the same source as our training set. We have designed our system for future enhancement of a robot platform that performs state-of-the-art object recognition and creates object maps of realistic environments. In this context, we demonstrate the use of spatial-semantic information to perform clustering and place labeling of object maps obtained from real homes. This place information is fed back into the robot system to inform an object search planner about likely locations of a query object. As a whole, this system represents a new level in spatial reasoning and semantic understanding for a physical platform.

Automated Place Classification Using Object Detection

“…Recognized objects and their locations can be used to automatically label places in the environment through the use of annotated databases, as demonstrated by our spatialsemantic modeling system [17]. We also demonstrated that the spatial-semantic model can be used to guide a robot to more productive locations during visual search.…”

“…This 3D data allows us to identify structures such as tables and desks and automatically segment and acquire multiple images of objects on their surface. We can also use the place labels to guide visual search of novel objects using the Location Model described in [17]. In addition, we need to investigate the use of place labels as context to enhance recognition of objects that are currently difficult to recognize.…”

“…This can be done based on image similarity as in [7]. Alternatively, our Cluster Model described in [17] can possibly be used to perform clustering based on both spatial and semantic information. Figure 6 shows the confusion matrix for the final place labeling of object clusters.…”

Topological Modeling using Sonar Grid Map

“…위 방법들과는 달리 비전 센서를 이용한 위치 표식(place labeling) 방법 [14] 이 이 용되었고, 하이브리드 지도의 전역 위상학적 지도 작성을 위한 비전 정보에 따른 노드 구성 방법도 제안되었다 [15] . 제안된 방법은 우리의 기존 방법을 기본으로 개발되었 다 [16,17] .…”

Extracting Semantic Information from Visual Data: A Survey

“…Once an object was observed, the semantic information was directly acquired. A similar work [93] built the spatial-semantic object models based on the LabelMe database [94]. Bayes' theorem was applied to define place categories by the recognized objects.…”