Deep Detection of People and their Mobility Aids for a Hospital Robot

Abstract

Robots operating in populated environments encounter many different types of people, some of whom might have an advanced need for cautious interaction, because of physical impairments or their advanced age. Robots therefore need to recognize such advanced demands to provide appropriate assistance, guidance or other forms of support. In this paper, we propose a depth-based perception pipeline that estimates the position and velocity of people in the environment and categorizes them according to the mobility aids they use: pedestrian, person in wheelchair, person in a wheelchair with a person pushing them, person with crutches and person using a walker. We present a fast region proposal method that feeds a Region-based Convolutional Network (Fast R-CNN [1]). With this, we speed up the object detection process by a factor of seven compared to a dense sliding window approach. We furthermore propose a probabilistic position, velocity and class estimator to smooth the CNN's detections and account for occlusions and misclassifications. In addition, we introduce a new hospital dataset with over 17,000 annotated RGB-D images. Extensive experiments confirm that our pipeline successfully keeps track of people and their mobility aids, even in challenging situations with multiple people from different categories and frequent occlusions. Videos of our experiments and the dataset are available at http://www2.informatik. uni-freiburg.de/˜kollmitz/MobilityAids. Fast R-CNN detection: Crutches Position: (1.63 m, 3.06 m) Velocity: 1.046 m/s HMM final class estimation: bg Fast R-CNN detection: Pedestrian Position: (1.71 m, 3.02 m) Velocity: 0.199 m/s HMM final class estimation: Fast R-CNN detection: ---Position: (1.29 m, 2.71 m) Velocity: 1.453 m/s HMM final class estimation: Fast R-CNN detection: Crutches Position: (1.62 m, 3.07 m) Velocity: 0.597 m/s HMM final class estimation: bg bg bg

Fast and Robust People Detection in RGB Images

“…Both Adam and AdamS led to poor performance on both datasets, creating many low-accuracy detections during the test phase. On the other hand, AdamW achieved good results, obtaining comparable results with Vasquez et al [47,63] on the MobilityAids dataset and acceptable results on the MOT17Det dataset. More about these results can be found in Section 5.…”

“…The first considered dataset is Mobility Aids, which was created and used by Vasquez et al [47]. It contains roughly 17,000 RGB, Depth, and DepthJet images grouped into sequences.…”

Remote Control of a Wheeled Robot by Visible Light for Support in Infectious Disease Hospitals

“…There have been previous studies of teleoperated mobile robots to assist with tasks in hospitals. However, most of those robots traveled on a predetermined path and performed a predetermined action [20]- [28], so they did not require much teleoperation. In addition, there have been reports of robots in the medical field that can be operated wirelessly, such as through the Internet [31], and Dallal et al [32] showed that a hospital mobile robot could be controlled over a wireless local area network.…”

Multi-Modal Detection and Mapping of Static and Dynamic Obstacles in Agriculture for Process Evaluation

“…One mathematical framework of generative models is the Hidden Markov Model (HMM) which is able to respect the time-domain and noisy sensor data of a process. Applications to robotics and grid maps have shown the incorporation of learning and decoding of hidden property information from the environment which makes HMMs a suitable approach to infer properties out of the semantical grid maps (Stachniss, 2009 ; Walter et al, 2013 ; Vasquez et al, 2017 ).…”