Navigation through urban environments by visual perception and interaction

Abstract

In the Autonomous City Explorer (ACE) project a mobile robot is developed, which is capable of finding its way to a given destination in an unknown urban environment. An exemplary mission is to find the way from our institute to the Marienplatz, a public place in the center of Munich, without any prior knowledge or GPS information. Inspired by the behavior of humans in unknown environments, ACE must find its way by asking pedestrians. The route is about 1.5 kilometers far and includes heavily traveled roads and crowded public places. In order to navigate safely in an unknown urban environment, some challenges arise for the vision system. Robust human detection, tracking and the estimation of human body poses is essential for natural interaction with pedestrians. Furthermore, the robot needs to be able to detect sidewalk and crossroads. A visual odometry system is used to support the conventional navigation. Outdoor experiments were conducted twice successfully. After about 5 hours and interacting with 25 and 38 persons respectively, ACE arrived the Marienplatz. This paper describes both, an architecture of the vision system used for ACE and the algorithms used to deal with the described challenges.

Robots asking for directions — The willingness of passers-by to support robots

“…Starting with the detected skin parts, the algorithm segments the point cloud into smaller clusters and fits a 28 degrees of freedom kinematic human model into the scene, thus retrieving the body pose and the direction of the pointing gesture. For further information on the Vision module please refer to [23].…”

The Autonomous City Explorer (ACE) project — mobile robot navigation in highly populated urban environments

“…For interaction with humans a touchscreen and speakers are used. A robotic head which comprises of five cameras and an animated mouth is mounted on the robot, for gesture recognition and people tracking [6]. More details on the hardware components can be found in [5].…”

“…The vision system of the robot is used to track the person [6]. The tracker runs at 10 Hz and its outputs are send to the Behavior Control.…”

“…On detection, a human is asked to touch the screen and point in the direction of its designated goal location [6]. The coherence of the information received during the interactions is checked within the HRI subsystem of the ACE robot [14].…”

“…It can be observed from the corresponding output (a2), that the planning module is inactive. The robot is directly send to the position of the person (red dot), which is received from the vision system [6].…”