Self-calibrating Marker Tracking in 3D with Event-Based Vision Sensors

Abstract

Abstract.Following an object's position relative to oneself is a fundamental functionality required in intelligent real-world interacting robotic systems. This paper presents a computationally efficient vision based 3D tracking system, which can ultimately operate in real-time on autonomous mobile robots in cluttered environments. At the core of the system, two neural inspired eventbased dynamic vision sensors (eDVS) independently track a high frequency flickering LED in their respective 2D angular coordinate frame. A self-adjusted feed-forward neural network maps those independent 2D angular coordinates into a Cartesian 3D position in world coordinates. During an initial calibration phase, an object composed of multiple independent markers with known geometry provides relative position information between those markers for network training (without ever using absolute world coordinates for training). In a subsequent application phase tracking a single marker yields position estimates relative to sensor origin, while tracking multiple markers provides additional orientation. The neural inspired vision-based tracking system runs in real-time on ARM7 microcontrollers, without the need for an external PC.

Event-Based Sensing and Signal Processing in the Visual, Auditory, and Olfactory Domain: A Review

“…The system is capable of detecting and tracking people within a 4 m range with a refresh rate of the depth map of up to 200 Hz. In another work (Müller and Conradt, 2012), two cameras independently track an object. Then a self-adjusted neural network maps the 2D angular coordinates into a Cartesian 3D position of the object.…”

Event-based Visual Servoing

“…In these works Ni et al introduce an event-based iterative closest point algorithm to track a microgripper's position at a high rate frequency. These dynamic vision sensors (eDVS) have also been used to track angular 2D coordinates frame to guide and operate in real-time autonomous mobile robots (Müller, 2012).…”

“…Other works like (Ni, 2012a) have used the event-based paradigm to track microparticles simply by means of a see and move strategy to track and operate mobile robots as in (Müller, 2012). Control loop has not visual feedback.…”

Asynchronous Event-based Cooperative Stereo Matching Using Neuromorphic Silicon Retinas

“…Since DVS sensors drastically reduce redundant pixels (e.g. static background features) and encode objects in a frame-less fashion with high temporal resolution (about 1 μs), it is well suited for fast motion analyses, tracking and surveillance [11][12][13][14][15]. These Sensors are capable of operating in uncontrolled environments with varying lighting conditions because of their high dynamic range of operation (120 dB).…”