Making Sense of Indoor Spaces Using Semantic Web Mining and Situated Robot Perception

Abstract

International audienceIntelligent Autonomous Robots deployed in human environments must have understanding of the wide range of possible semantic identities associated with the spaces they inhabit – kitchens, living rooms, bathrooms, offices, garages, etc. We believe robots should learn this information through their own exploration and situated perception in order to uncover and exploit structure in their environments – structure that may not be apparent to human engineers, or that may emerge over time during a deployment. In this work, we combine semantic web-mining and situated robot perception to develop a system capable of assigning semantic categories to regions of space. This is accomplished by looking at web-mined relationships between room categories and objects identified by a Convolutional Neural Network trained on 1000 categories. Evaluated on real-world data, we show that our system exhibits several conceptual and technical advantages over similar systems, and uncovers semantic structure in the environment overlooked by ground-truth annotators

A Knowledge Retrieval Framework for Household Objects and Actions with External Knowledge

“…Our semantic matching algorithm was mostly inspired by the works of Young et al [35], and Icarte et al [12] where they use CS knowledge from the web ontologies DBpedia, ConceptNet, and WordNet to find the label of unknown objects. As well as from the studies [6,36], where the label of the room can be understood through the objects that the cognitive robotic system perceived from its vision module. One drawback that can be noticed in these works, is that all of them depend on only one ontology.…”

“…One drawback that can be noticed in these works, is that all of them depend on only one ontology. Young et al compares only the DBpedia comment boxes between the entities, Icarte et al acquires only the property values from ConceptNet of the entities, and [ 6 , 36 ] on the synonyms, hypernyms, and hyponyms of WordNet entities.…”

Human-centric partitioning of the environment

“…The second group consists of global approaches where the aim is to partition a complete floor plan rather than single scenes [9], [10], [11], [12], [13], [14]. For the global approaches, various methods exist such as Voronoi partitioning, feature-based methods and segmentation based methods [13].…”

“…Natural language descriptions along with topological and semantic representations are used to semantically label an indoor environment in [12]. In, [14] office floors are semantically partitioned into 3 categories that are office, kitchen or eating area. This approach uses the locations of recognized objects together with semantic web mining to infer the category of the scenes observed by the robot at different waypoints.…”

Adding Uncertainty to an Object Detection System for Mobile Robots

“…Associating objects of daily use with certain categories of places facilitates the search for an object in a specific room context among other tasks [58,191]. In the same way, some objects tend to be near or far from others.…”