Semantic Mapping and Object Detection for Indoor Mobile Robots

Kowalewski, Stefan; Maurin, Adrian Llopart; Andersen, Jens Christian

doi:10.1088/1757-899x/517/1/012012

Cited by 7 publications

(3 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [ 24 ], Doan et al proposed a semantic segmentation network with residual depth-wise separable blocks to detect street objects such as cars and pedestrians. In [ 25 ], Kowalewski et al presented the object-level semantic perception of the environment for indoor mobile robots. The experiments results indicated that the proposed framework, the Mask-RCNN, achieved the mAP score of 0.414.…”

Section: Related Workmentioning

confidence: 99%

Object-of-Interest Perception in a Reconfigurable Rolling-Crawling Robot

Semwal

Lee

Sánchez

et al. 2022

Sensors

View full text Add to dashboard Cite

Cebrenus Rechenburgi, a member of the huntsman spider family have inspired researchers to adopt different locomotion modes in reconfigurable robotic development. Object-of-interest perception is crucial for such a robot to provide fundamental information on the traversed pathways and guide its locomotion mode transformation. Therefore, we present a object-of-interest perception in a reconfigurable rolling-crawling robot and identifying appropriate locomotion modes. We demonstrate it in Scorpio, our in-house developed robot with two locomotion modes: rolling and crawling. We train the locomotion mode recognition framework, named Pyramid Scene Parsing Network (PSPNet), with a self-collected dataset composed of two categories paths, unobstructed paths (e.g., floor) for rolling and obstructed paths (e.g., with person, railing, stairs, static objects and wall) for crawling, respectively. The efficiency of the proposed framework has been validated with evaluation metrics in offline and real-time field trial tests. The experiment results show that the trained model can achieve an mIOU score of 72.28 and 70.63 in offline and online testing, respectively for both environments. The proposed framework’s performance is compared with semantic framework (HRNet and Deeplabv3) where the proposed framework outperforms in terms of mIOU and speed. Furthermore, the experimental results has revealed that the robot’s maneuverability is stable, and the proposed framework can successfully determine the appropriate locomotion modes with enhanced accuracy during complex pathways.

show abstract

Section: Related Workmentioning

confidence: 99%

Object-of-Interest Perception in a Reconfigurable Rolling-Crawling Robot

Semwal

Lee

Sánchez

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…21 After employing Mask-RCNN 6 for image segmentation, Kowalewski et al presented a full solution that produces object-level semantic perception of the environment for indoor mobile robot. 22 Although the visionbased detectors show advances in robot vision, many of them tend to assemble techniques.…”

Section: Related Workmentioning

confidence: 99%

Geometric property-based convolutional neural network for indoor object detection

Ding

Wang

2021

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

Indoor object detection is a very demanding and important task for robot applications. Object knowledge, such as two-dimensional (2D) shape and depth information, may be helpful for detection. In this article, we focus on region-based convolutional neural network (CNN) detector and propose a geometric property-based Faster R-CNN method (GP-Faster) for indoor object detection. GP-Faster incorporates geometric property in Faster R-CNN to improve the detection performance. In detail, we first use mesh grids that are the intersections of direct and inverse proportion functions to generate appropriate anchors for indoor objects. After the anchors are regressed to the regions of interest produced by a region proposal network (RPN-RoIs), we then use 2D geometric constraints to refine the RPN-RoIs, in which the 2D constraint of every classification is a convex hull region enclosing the width and height coordinates of the ground-truth boxes on the training set. Comparison experiments are implemented on two indoor datasets SUN2012 and NYUv2. Since the depth information is available in NYUv2, we involve depth constraints in GP-Faster and propose 3D geometric property-based Faster R-CNN (DGP-Faster) on NYUv2. The experimental results show that both GP-Faster and DGP-Faster increase the performance of the mean average precision.

show abstract

“…In studies on the detection of objects inside buildings [7,8], an artificial intelligence model studies objects designated by labels and detects them. Meanwhile, studies on obstacle detection [9][10][11] detect obstacles in inner spaces of buildings such as rooms or plazas. Studies on passages inside buildings such as corridors focus on predicting the path by identifying the characteristics of passages rather than detecting obstacles [12][13][14].…”

Section: Introductionmentioning

confidence: 99%

Vision-based Recognition Algorithm for Up-To-Date Indoor Digital Map Generations at Damaged Buildings

Kim¹,

Kim²,

Ahn³

2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

When firefighters are engaged in search and rescue missions inside a building at a risk of collapse, they have difficulty in field command and rescue because they can only simply monitor the situation inside the building utilizing old building drawings or robots. To propose an efficient solution for fast search and rescue work of firefighters, this study investigates the generation of up-to-date digital maps for disaster sites by tracking the collapse situation, and identifying the information of obstacles which are risk factors, using an artificial intelligence algorithm based on low-cost robots. Our research separates the floor by using the mask regional convolutional neural network (R-CNN) algorithm, and determines whether the passage is collapsed or not. Then, in the case of a passage that can be searched, the floor pattern of the obstacles that exist on the floor that has not collapsed is analyzed, and obstacles are searched utilizing an image processing algorithm. Here, we can detect various unknown as well as known obstacles. Furthermore, the locations of obstacles can be estimated using the pixel values up to the bounding box of an existing detected obstacle. We conduct experiments using the public datasets collected by Carnegie Mellon university (CMU) and data collected by manipulating a low-cost robot equipped with a smartphone while roaming five buildings in a campus. The collected data have various floor patterns for objectivity and obstacles that are different from one another. Based on these data, the algorithm for detecting unknown obstacles of a verified study and estimating their sizes had an accuracy of 93%, and the algorithm for estimating the distance to obstacles had an error rate of 0.133. Through this process, we tracked collapsed passages and composed up-to-date digital maps for disaster sites that include the information of obstacles that interfere with the search and rescue work.

show abstract

Semantic Mapping and Object Detection for Indoor Mobile Robots

Cited by 7 publications

References 1 publication

Object-of-Interest Perception in a Reconfigurable Rolling-Crawling Robot

Object-of-Interest Perception in a Reconfigurable Rolling-Crawling Robot

Geometric property-based convolutional neural network for indoor object detection

Vision-based Recognition Algorithm for Up-To-Date Indoor Digital Map Generations at Damaged Buildings

Contact Info

Product

Resources

About