Real-time 2D–3D door detection and state classification on a low-power device

Abstract

In this paper, we propose three methods for door state classification with the goal to improve robot navigation in indoor spaces. These methods were also developed to be used in other areas and applications since they are not limited to door detection as other related works are. Our methods work offline, in low-powered computers as the Jetson Nano, in real-time with the ability to differentiate between open, closed and semi-open doors. We use the 3D object classification, PointNet, real-time semantic segmentation algorithms such as, FastFCN, FC-HarDNet, SegNet and BiSeNet, the object detection algorithm, DetectNet and 2D object classification networks, AlexNet and GoogleNet. We built a 3D and RGB door dataset with images from several indoor environments using a 3D Realsense camera D435. This dataset is freely available online. All methods are analysed taking into account their accuracy and the speed of the algorithm in a low powered computer. We conclude that it is possible to have a door classification algorithm running in real-time on a low-power device.

Towards Fully Autonomous UAV: Damaged Building-Opening Detection for Outdoor-Indoor Transition in Urban Search and Rescue

“…Previous work in opening detection presented three stages to detect and classify door openings using RGB and depth information [24]. That algorithm was used to classify a door into different opening classes, i.e., open, semi-open, and closed, successfully classifying doors in real time on a limited powered device used for indoor navigation purposes to navigate between rooms.…”

“…That algorithm was used to classify a door into different opening classes, i.e., open, semi-open, and closed, successfully classifying doors in real time on a limited powered device used for indoor navigation purposes to navigate between rooms. The network developed in [24] was based on a sequential method that combined object detection with a classification network to detect and segment doors. The object detection allowed the cropping of parts of the initial images, that were then used as input to classify open, semi-open, or closed doors.…”

Introduction to Door Opening Type Classification Based on Human Demonstration

“…In [ 22 ], Chen et al proposed a convolutional neural network (CNN) that requires only one RGB image to estimate pose of doors. Ramôa et al [ 23 ] proposed multiple 2D and 3D methods for door detection and state classification that can be used in real time on a low-power device. They used PointNet for 3D door state classification and compared multiple neural networks for 2D detection and classification.…”

Enhancing Door Detection for Autonomous Mobile Robots with Environment-Specific Data Collection

“…To evaluate the methods introduced in this work, we start by performing a set of preliminary experiments to assess the performance of the general door detector described in Section IV-A on a publicly available dataset used to test door detection methods from RGB images, which is less challenging than our proposed one. We exploit the DeepDoors2 dataset [28], that contains 3000 labelled RGB images of doors with different textures and sizes. The dataset contains images of doors that are sometimes obstructed by obstacles (e.g., furniture or persons) but the labels are assigned only for those doors that are totally contained in the frame and close enough to be sharply distinguishable.…”