Using Deep Learning to Find Victims in Unknown Cluttered Urban Search and Rescue Environments

Abstract

Purpose of Review We investigate the first use of deep networks for victim identification in Urban Search and Rescue (USAR). Moreover, we provide the first experimental comparison of single-stage and two-stage networks for body part detection, for cases of partial occlusions and varying illumination, on a RGB-D dataset obtained by a mobile robot navigating cluttered USAR-like environments. Recent Findings We considered the single-stage detectors Single Shot Multi-box Detector, You Only Look Once, and RetinaNet and the two-stage Feature Pyramid Network detector. Experimental results show that RetinaNet has the highest mean average precision (77.66%) and recall (86.98%) for detecting victims with body part occlusions in different lighting conditions. Summary End-to-end deep networks can be used for finding victims in USAR by autonomously extracting RGB-D image features from sensory data. We show that RetinaNet using RGB-D is robust to body part occlusions and low-lighting conditions and outperforms other detectors regardless of the image input type.

Visual victim detection and quadrotor-swarm coordination control in search and rescue environment

“…The image processing task is performed by a CNN, which is in charge of image analysis focusing on the identification of the potential human victims. As shown in [30] the victims are presented in a urban search and rescue like environments thus the victims will be under rubble or some kind of object will obstruct a percentage of the victim so is important to train a CNN with this kind of data in order to get a better detection, but to get the amount of images that meet this characteristics is a hard task Ì ISSN: 2088-8708 by itself, in the paper they create a set of 570 images, so it can be consider a small dataset. Transfer learning is a quite useful method for object detection where a feature extractor is use then top layers for classification are fine tune for the task in hand, the need of a dataset that can generalize a desire concept could be difficult to acquire so a virtual environment can be use due to the fact that is more flexible, labels or bounding boxes can be extracted in a automated manner and we can get models of any existing object if needed, in [31] the aim of the paper is to combine this two methods using transfer learning and a virtual dataset to get a pedestrian detector, the results show that high performance can be achieve in real world datasets when the training was done on a virtual dataset then a small set of real images are used for fine tuning.…”

Autonomous Thermal Vision Robotic System for Victims Recognition in Search and Rescue Missions

“…The main developments for people detection in Search and Rescue scenarios using robots, focus on using RGB cameras (additive color mode Red, Green, and Blue) [23][24][25]. The methods based on thermal images are very limited to classical computer vision techniques applications and primitive neural networks.…”