ÐWe describe a framework that explicitly reasons about data association to improve tracking performance in many difficult visual environments. A hierarchy of tracking strategies results from ascribing ambiguous or missing data to: 1) noise-like visual occurrences, 2) persistent, known scene elements (i.e., other tracked objects), or 3) persistent, unknown scene elements. First, we introduce a randomized tracking algorithm adapted from an existing probabilistic data association filter (PDAF) that is resistant to clutter and follows agile motion. The algorithm is applied to three different tracking modalitiesÐhomogeneous regions, textured regions, and snakesÐand extensibly defined for straightforward inclusion of other methods. Second, we add the capacity to track multiple objects by adapting to vision a joint PDAF which oversees correspondence choices between same-modality trackers and image features. We then derive a related technique that allows mixed tracker modalities and handles object overlaps robustly. Finally, we represent complex objects as conjunctions of cues that are diverse both geometrically (e.g., parts) and qualitatively (e.g., attributes). Rigid and hinge constraints between part trackers and multiple descriptive attributes for individual parts render the whole object more distinctive, reducing susceptibility to mistracking. Results are given for diverse objects such as people, microscopic cells, and chess pieces.

Many rural roads lack sharp, smoothly curving edges and a homogeneous surface appearance, hampering traditional vision-based road-following methods. However, they often have strong texture cues parallel to the road direction in the form of ruts and tracks left by other vehicles. In this paper, we describe an algorithm for following ill-structured roads in which dominant texture orientations computed with multi-scale Gabor wavelet filters vote for a consensus road vanishing point location. In-plane road curvature and out-of-plane undulation are estimated in each image by tracking the vanishing point indicated by a horizontal image strip as it moves up toward the putative vanishing line. Particle filtering is also used to track the vanishing point sequence induced by road curvature from image to image. Results are shown for vanishing point localization on a variety of road scenes ranging from gravel roads to dirt trails to highways.

We describe results on combining depth information from a laser range-finder and color and texture image cues to segment ill-structured dirt, gravel, and asphalt roads as input to an autonomous road following system. A large number of registered laser and camera images were captured at frame-rate on a variety of rural roads, allowing laser features such as 3-D height and smoothness to be correlated with image features such as color histograms and Gabor filter responses. A small set of road models was generated by training separate neural networks on labeled feature vectors clustered by road "type." By first classifying the type of a novel road image, an appropriate second-stage classifier was selected t o segment individual pixels, achieving a high degree of accuracy on arbitrary images from the dataset. Segmented images combined with laser range information and the vehicle's inertial navigation data were used to construct 3-D maps suitable for path planning.

Many rural roads lack sharp, smoothly curving edges and a homogeneous surface appearance, hampering traditional vision-based road-following methods. However, they often have strong texture cues parallel to the road direction in the form of ruts and tracks left by other vehicles. This paper describes an unsupervised algorithm for following ill-structured roads in which dominant texture orientations computed with Gabor wavelet filters vote for a consensus road vanishing point location. The technique is first described for estimating the direction of straight-road segments, then extended to curved and undulating roads by tracking the vanishing point indicated by a differential "strip" of voters moving up toward the nominal vanishing line. Finally, the vanishing point is used to constrain a search for the road boundaries by maximizing texture-and color-based region discriminant functions. Results are shown for a variety of road scenes including gravel roads, dirt trails, and highways.

We present a general system with a focus on addressing three events of the 2013 DARPA Robotics Challenge (DRC) trials: debris clearing, door opening, and wall breaking. Our hardware platform is DRC‐HUBO, a redesigned model of the HUBO2+ humanoid robot developed by KAIST and Rainbow, Inc. Our system allowed a trio of operators to coordinate a 32 degree‐of‐freedom robot on a variety of complex mobile manipulation tasks using a single, unified approach. In addition to descriptions of the hardware and software, and results as deployed on the DRC‐HUBO platform, we present some qualitative analysis of lessons learned from this demanding and difficult challenge.