An Indoor Room Classification System for Social Robots via Integration of CNN and ECOC

Abstract

The ability to classify rooms in a home is one of many attributes that are desired for social robots. In this paper, we address the problem of indoor room classification via several convolutional neural network (CNN) architectures, i.e., VGG16, VGG19, & Inception V3. The main objective is to recognize five indoor classes (bathroom, bedroom, dining room, kitchen, and living room) from a Places dataset. We considered 11600 images per class and subsequently fine-tuned the networks. The simulation studies suggest that cleaning the disparate data produced much better results in all the examined CNN architectures. We report that VGG16 & VGG19 fine-tuned models with training on all layers produced the best validation accuracy, with 93.29% and 93.61% on clean data, respectively. We also propose and examine a combination model of CNN and a multi-binary classifier referred to as error correcting output code (ECOC) with the clean data. The highest validation accuracy of 15 binary classifiers reached up to 98.5%, where the average of all classifiers was 95.37%. CNN and CNN-ECOC, and an alternative form called CNN-ECOC Regression, were evaluated in real-time implementation on a NAO humanoid robot. The results show the superiority of the combination model of CNN and ECOC over the conventional CNN. The implications and the challenges of real-time experiments are also discussed in the paper.

A Doorway Detection and Direction (3Ds) System for Social Robots via a Monocular Camera

“…Whereas this task does not require much effort for humans and even their pets, it is a challenge for social and other autonomous robots. As such, it is desired that social robots have the same skill and are able to move around a house seamlessly and know their own whereabouts based on an ability to classify each room and its functionality [1,2]. Indoor navigation is inherently multifaceted and includes several tasks including but not limited to localization, mapping, Simultaneous Localization and Mapping (SLAM), path planning, object, and scene recognition.…”

Sequential Localizing and Mapping: A Navigation Strategy via Enhanced Subsumption Architecture

“…The proposed system is designed and tested with few assumptions. First, the five different types of rooms are assigned for the CNN model with labelled images, in which the corridor is not one of the classes (see [ 37 ]). Second, we assume the practical predictions of classifying the room (see [ 38 ]) and detecting the doorway (see [ 54 ]) are always true positive within the navigation task.…”

“…The achievement task is to end the navigation process via following a voice command made by a companion using Nao’s speaker. The “Command Detection” perception module can be designed by detecting one of the five room classes from room classification component, as presented in [ 37 ]. By comparing the command and the information in the knowledge system, the “Sitting Down” action module will be run as an indication of completing the navigation process.…”

“…Room Localization and Doorway Detection Modules: (CNN-Based Models): the knowledge-based system aims to create a Directional Semantic Topological Map via designing a module called “Topo-Room”, which will be explained below. Therefore, the system has to begin with “Room Localization” and “Doorway Detection” perception modules that successfully designed and tested, using the CNN model as explained in detail in [ 37 , 38 , 54 ], respectively. “Room Localization” provides the semantic feature, whereas “Doorway Detection” provides the directional feature to the topological map.…”

“…With the proposed behavioristic system, associated issues to the exploration and SLAM with classical techniques will be addressed or improved. For example, the semantic information within the DST-Map helps SLAM via recognizing the scene and classifying the type of the room [ 37 , 38 ] which gives a meaningful localization and mapping. Consequently, other robotic applications will be improved because of the meaningful task such as the interaction between humans and robots or switching the planning function from a path issue to a task issue, e.g., the robot needs to go to the bedroom.…”