Semantic Map Augmentation for Robot Navigation: A Learning Approach Based on Visual and Depth Data

Bersan, Dhiego; Martins, Renato; Campos, Mário F. M.

doi:10.1109/lars/sbr/wre.2018.00018

Cited by 15 publications

(16 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the door transition points, the pixels are searched on the depth map obtained from the image with deep learning approaches. Bersan, Martins, Campos, and Nascimento (2018) used 2D CNN-based YOLO (Redmon, Divvala, Girshick, and Farhadi, 2016) object identifier and 3D model-based segmentation algorithm for door object during semantic metric mapping of the environment. The point cloud was obtained using the depth information of the pixels in the bounding box with YOLO, and only the points belonging to the door were extracted using RANSAC.…”

Section: Learning-based Approachesmentioning

confidence: 99%

Iç Ortamlarda Kapilarin Tespi̇ti̇ İçi̇n Deri̇n Öğrenme Tekni̇kleri̇ni̇n Karşilaştirilmasi

Kaleci

Turgut

2021

Eskişehir Osmangazi Üniversitesi Mühendislik Ve Mimarlık Fakültesi Dergisi

View full text Add to dashboard Cite

In indoor environments, the detection of doors (open, semi-opened, and closed) is a crucial task for a variety of fields such as robotics, computer vision, and architecture. The studies that are addressed the door detection problem can be divided into three major categories: 1) closed doors via visual data, 2) open doors via range data, and 3) open, semi-opened, and closed doors via point cloud data. Although some successful studies have been proposed being detected doors via visual and range data under specific circumstances, in this study, we exploited point cloud data due to its ability to describe the 3D characteristic of scenes. The main contribution of this study is two-fold. Firstly, we mainly intended to discover the potential of point-based deep learning architectures such as PointNet, PointNet++, Dynamic Graph Convolutional Neural Network (DGCNN), PointCNN, and Point2Sequence, in contrast to previous studies that generally defined a set of rules depending on the type of door and characteristics of the data. Secondly, the OGUROB DOORS dataset is constructed, which contains point cloud data captured in the GAZEBO simulation environment in different robot positions and orientations. We used precision, recall, and F1-score metrics to analyze the merit and demerit aspects of these architectures. Also, some visual results were given to describe the characteristics of these architectures. The test results showed that all architectures are capable of classifying open, semi-opened, and closed doors over 98% accuracy.

show abstract

Section: Learning-based Approachesmentioning

confidence: 99%

Iç Ortamlarda Kapilarin Tespi̇ti̇ İçi̇n Deri̇n Öğrenme Tekni̇kleri̇ni̇n Karşilaştirilmasi

Kaleci

Turgut

2021

Eskişehir Osmangazi Üniversitesi Mühendislik Ve Mimarlık Fakültesi Dergisi

View full text Add to dashboard Cite

show abstract

“…All the resulted associations which distances are smaller than a threshold (D(x i , y j ) < δ) are assumed to correspond to previously seen objects; otherwise, new object instances representing the remaining observations are included in the dictionary. In order to track and to increase the accuracy of detected instances, each stored semantic object is modeled with a constant state Kalman filter [6], since we are interested in storing mostly static classes in the final augmented map, to maintain its state up-to-date and combine different objects observations. Each filter combines the information of the different observations temporally as shown in Figure 8.…”

Section: Object Tracking and Final Augmented Representationmentioning

confidence: 99%

“…A preliminary conference version paper is introduced in our previous [6]. In this manuscript, we have made a number of major modifications that we summarize as follows: -The localization and object tracking of the classes are improved to handle multiple objects per image and to support online pose updates, during loop closing of the localization and metric mapping back-end.…”

Section: Introductionmentioning

confidence: 99%

Extending Maps with Semantic and Contextual Object Information for Robot Navigation: a Learning-Based Framework Using Visual and Depth Cues

Martins

Bersan

Campos

et al. 2020

J Intell Robot Syst

Self Cite

View full text Add to dashboard Cite

This paper addresses the problem of building augmented metric representations of scenes with semantic information from RGB-D images. We propose a complete framework to create an enhanced map representation of the environment with object-level information to be used in several applications such as human-robot interaction, assistive robotics, visual navigation, or in manipulation tasks. Our formulation leverages a CNN-based object detector (Yolo) with a 3D model-based segmentation technique to perform instance semantic segmentation, and to localize, identify, and track different classes of objects in the scene. The tracking and positioning of semantic classes is done with a dictionary of Kalman filters in order to combine sensor measurements over time and then providing more accurate maps. The formulation is designed to identify and to disregard dynamic objects in order to obtain a mediumterm invariant map representation. The proposed method was evaluated with collected and publicly available RGB-D data sequences acquired in different indoor scenes. Experimental results show the potential of the technique to produce augmented semantic maps containing several objects (notably doors). We also provide to the community a dataset composed of annotated object classes (doors, fire extinguishers, benches, water fountains) and their positioning, as well as the source code as ROS packages. 1 1 Preprint paper version to appear at Journal of Intelligent & Robotic Systems, available online at: https://doi.

show abstract

“…where Steering represents the value in px of the worst case, and the place variable includes the "g" value for when the maximum value is detected. This steering value is transformed into a percentage value by considering the limitation indicated by Equations (17) and (18):…”

Section: Path Calculation and Motionmentioning

confidence: 99%

Semantic Segmentation to Develop an Indoor Navigation System for an Autonomous Mobile Robot

et al. 2020

View full text Add to dashboard Cite

In this study, a semantic segmentation network is presented to develop an indoor navigation system for a mobile robot. Semantic segmentation can be applied by adopting different techniques, such as a convolutional neural network (CNN). However, in the present work, a residual neural network is implemented by engaging in ResNet-18 transfer learning to distinguish between the floor, which is the navigation free space, and the walls, which are the obstacles. After the learning process, the semantic segmentation floor mask is used to implement indoor navigation and motion calculations for the autonomous mobile robot. This motion calculations are based on how much the estimated path differs from the center vertical line. The highest point is used to move the motors toward that direction. In this way, the robot can move in a real scenario by avoiding different obstacles. Finally, the results are collected by analyzing the motor duty cycle and the neural network execution time to review the robot’s performance. Moreover, a different net comparison is made to determine other architectures’ reaction times and accuracy values.

show abstract

Semantic Map Augmentation for Robot Navigation: A Learning Approach Based on Visual and Depth Data

Cited by 15 publications

References 15 publications

Iç Ortamlarda Kapilarin Tespi̇ti̇ İçi̇n Deri̇n Öğrenme Tekni̇kleri̇ni̇n Karşilaştirilmasi

Iç Ortamlarda Kapilarin Tespi̇ti̇ İçi̇n Deri̇n Öğrenme Tekni̇kleri̇ni̇n Karşilaştirilmasi

Extending Maps with Semantic and Contextual Object Information for Robot Navigation: a Learning-Based Framework Using Visual and Depth Cues

Semantic Segmentation to Develop an Indoor Navigation System for an Autonomous Mobile Robot

Contact Info

Product

Resources

About