Robots operating in populated environments encounter many different types of people, some of whom might have an advanced need for cautious interaction, because of physical impairments or their advanced age. Robots therefore need to recognize such advanced demands to provide appropriate assistance, guidance or other forms of support. In this paper, we propose a depth-based perception pipeline that estimates the position and velocity of people in the environment and categorizes them according to the mobility aids they use: pedestrian, person in wheelchair, person in a wheelchair with a person pushing them, person with crutches and person using a walker. We present a fast region proposal method that feeds a Region-based Convolutional Network (Fast R-CNN [1]). With this, we speed up the object detection process by a factor of seven compared to a dense sliding window approach. We furthermore propose a probabilistic position, velocity and class estimator to smooth the CNN's detections and account for occlusions and misclassifications. In addition, we introduce a new hospital dataset with over 17,000 annotated RGB-D images. Extensive experiments confirm that our pipeline successfully keeps track of people and their mobility aids, even in challenging situations with multiple people from different categories and frequent occlusions. Videos of our experiments and the dataset are available at http://www2.informatik. uni-freiburg.de/˜kollmitz/MobilityAids. Fast R-CNN detection: Crutches Position: (1.63 m, 3.06 m) Velocity: 1.046 m/s HMM final class estimation: bg Fast R-CNN detection: Pedestrian Position: (1.71 m, 3.02 m) Velocity: 0.199 m/s HMM final class estimation: Fast R-CNN detection: ---Position: (1.29 m, 2.71 m) Velocity: 1.453 m/s HMM final class estimation: Fast R-CNN detection: Crutches Position: (1.62 m, 3.07 m) Velocity: 0.597 m/s HMM final class estimation: bg bg bg
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.