Stereo-vision-based perception capabilities developed during the Robotics Collaborative Technology Alliances program

Abstract

The Robotics Collaborative Technology Alliances (RCTA) program, which ran from 2001 to 2009, was funded by the U.S. Army Research Laboratory and managed by General Dynamics Robotic Systems. The alliance brought together a team of government, industrial, and academic institutions to address research and development required to enable the deployment of future military unmanned ground vehicle systems ranging in size from man-portables to ground combat vehicles. Under RCTA, three technology areas critical to the development of future autonomous unmanned systems were addressed: advanced perception, intelligent control architectures and tactical behaviors, and human-robot interaction. The Jet Propulsion Laboratory (JPL) participated as a member for the entire program, working four tasks in the advanced perception technology area: stereo improvements, terrain classification, pedestrian detection in dynamic environments, and long range terrain classification. Under the stereo task, significant improvements were made to the quality of stereo range data used as a front end to the other three tasks. Under the terrain classification task, a multi-cue water detector was developed that fuses cues from color, texture, and stereo range data, and three standalone water detectors were developed based on sky reflections, object reflections (such as trees), and color variation. In addition, a multi-sensor mud detector was developed that fuses cues from color stereo and polarization sensors. Under the long range terrain classification task, a classifier was implemented that uses unsupervised and self-supervised learning of traversability to extend the classification of terrain over which the vehicle drives to the far-field. Under the pedestrian detection task, stereo vision was used to identify regions-of-interest in an image, classify those regions based on shape, and track detected pedestrians in three-dimensional world coordinates. To improve the detectability of partially occluded pedestrians and reduce pedestrian false alarms, a vehicle detection algorithm was developed. This paper summarizes JPL's stereo-vision based perception contributions to the RCTA program.

Advances in Space Robotics Autonomy

“…For example, during the 2001-2009 U.S. Army Research Laboratory (ARL) Robotics Collaborative Technology Alliances (RCTA) program, stereo improvements came from use of multiple baselines, higher resolution images, multiresolution stereo processing, and from algorithmic improvements to each stage of the stereo pipeline (11). In this section, we summarize recent improvements made to JPL's dense stereo ranging algorithms.…”

Evaluating the performance of unmanned ground vehicle water detection

“…Under the Robotics Collaborative Technology Alliances (RCTA) program, several researchers developed methods for water detection under the advanced perception technology thrust [1][2] [3][4] [5][6] [7].…”

Vision-based traversability estimation in field environments

“…Many autonomous systems consider obstacles as the only hazards present in the environment (Kim, Oh, & Rehg, 2007;Rankin et al, 2010;Thrun et al, 2007;Upcroft et al, 2007). Obstacles are regions which the robot must avoid.…”

“…The most common solution to obstacle detection is the use of structure cues, often simply labelling all points within a height range as an obstacle (Hadsell et al, 2009;Hadsell et al, 2007;Rankin, et al, 2010;Thrun, et al, 2007;Upcroft, et al, 2007;Vernaza, Taskar, & Lee, 2008). This method is adequate for many indoor environments due to their highly structured nature; traversable regions are invariably flat ground, while any protrusion from the ground can be considered an obstacle.…”

“…Positive and negative obstacles can be detected in 3D data using a known ground plane, and then locating points which are above and below the ground respectively by a given threshold (Hadsell, et al, 2009;Hadsell, et al, 2007;Rankin, et al, 2010;Thrun, et al, 2007;Upcroft, et al, 2007;Vernaza, et al, 2008).…”