Performance measurements for evaluating static and dynamic multiple human detection and tracking systems in unstructured environments

Abstract

The Army Research Laboratory (ARL) Robotics Collaborative Technology Alliance (CTA) conducted an assessment and evaluation of multiple algorithms for real-time detection of pedestrians in Laser Detection and Ranging (LADAR) and video sensor data taken from a moving platform. The algorithms were developed by Robotics CTA members and then assessed in field experiments jointly conducted by the National Institute of Standards and Technology (NIST) and ARL. A robust, accurate and independent pedestrian tracking system was developed to provide ground truth. The ground truth was used to evaluate the CTA member algorithms for uncertainty and error in their results. A real-time display system was used to provide early detection of errors in data collection.

3D Ground-Truth Systems for Object/Human Recognition and Tracking

“…The UWB tracking system [12,13] uses a collection of UWB radio receivers located around the perimeter of the test area to track multiple static and dynamic objects with credit-card size transmitter tags (see Figure 1). Each tag sends UWB pulses, which are detected by the antenna using a combination of TDOA (Time-Difference of Arrival) and AOA (Angle of Arrival) techniques to estimate the 2D or 3D position of the tags, dependent upon the test.…”

“…UWB is robust and provides higher precision indoor positioning compared to other wireless technologies. These systems can cover a very large area compared to other technologies and have been used successfully for human-and objecttracking applications [12,13].…”

“…We have used UWB to track vehicles and personnel throughout an area over 80,000 m 2 with an average error of 23 cm. The system has an update rate of 25 Hz to 50 Hz, which is sufficient to track vehicles at highway speeds [12,13]. We have also used it to track robots through random mazes with plywood walls (non-line-of-sight) achieving similar performance.…”

“…The algorithms used LADAR (laser detection and ranging) and video sensors mounted on a moving platform [12,13]. For the evaluation, the moving platform was a robot vehicle equipped with two pairs of stereo cameras, two sets of imaging LADARs and two sets of 2D laser line scanners.…”

Performance Metrics for Evaluating Object and Human Detection and Tracking Systems

“…NIST has worked with the United States (US) Department of Homeland Security, with the British Home Office, and with the European CHIL [14] program and the CLEAR [3] evaluations. NIST has also worked with the US Army Collaborative Technology Alliance (CTA) on Robotics to evaluate systems that locate and track human pedestrians from a moving vehicle [15].…”