A pencil balancing robot using a pair of AER dynamic vision sensors

2014 IEEE/RSJ International Conference on Intelligent Robots and Systems

Huber

Scaramuzza

2014

196

167

Abstract-In the last few years, we have witnessed impressive demonstrations of aggressive flights and acrobatics using quadrotors. However, those robots are actually blind. They do not see by themselves, but through the "eyes" of an external motion capture system. Flight maneuvers using onboard sensors are still slow compared to those attainable with motion capture systems. At the current state, the agility of a robot is limited by the latency of its perception pipeline. To obtain more agile robots, we need to use faster sensors. In this paper, we present the first onboard perception system for 6-DOF localization during high-speed maneuvers using a Dynamic Vision Sensor (DVS). Unlike a standard CMOS camera, a DVS does not wastefully send full image frames at a fixed frame rate. Conversely, similar to the human eye, it only transmits pixel-level brightness changes at the time they occur with microsecond resolution, thus, offering the possibility to create a perception pipeline whose latency is negligible compared to the dynamics of the robot. We exploit these characteristics to estimate the pose of a quadrotor with respect to a known pattern during high-speed maneuvers, such as flips, with rotational speeds up to 1,200• /s. Additionally, we provide a versatile method to capture ground-truth data using a DVS.

Section: B Related Workmentioning

confidence: 99%

Event-based, 6-DOF pose tracking for high-speed maneuvers

2014 IEEE/RSJ International Conference on Intelligent Robots and Systems

Huber

Scaramuzza

2014

196

167

“…This allows us to incorporate the measurement noise in pixel units with the dynamics of the ball (8). The coordinates in image space and world coordinates are linked by the pinhole camera model, i.e.,…”

Section: Extended Kalman Filtermentioning

confidence: 99%

“…For the covariance transformation, we use a first-order approximation. We then use the dynamical model (8) to propagate the system in world coordinates. We define the world-coordinate origin to coincide with the quadrotor center.…”

Section: E Trajectory Propagationmentioning

confidence: 99%

Towards evasive maneuvers with quadrotors using dynamic vision sensors

2015 European Conference on Mobile Robots (ECMR)

Baumli

Fontana

et al. 2015

We present a method to predict collisions with objects thrown at a quadrotor using a pair of dynamic vision sensors (DVS). Due to the micro-second temporal resolution of these sensors and the sparsity of their output, the object's trajectory can be estimated with minimal latency. Unlike standard cameras that send frames at a fixed frame rate, a DVS only transmits pixel-level brightness changes ("events") at the time they occur. Our method tracks spherical objects on the image plane using probabilistic trackers that are updated with each incoming event. The object's trajectory is estimated using an Extended Kalman Filter with a mixed state space that allows incorporation of both the object's dynamics and the measurement noise in the image plane. Using error-propagation techniques, we predict a collision if the 3-ellipsoid along the predicted trajectory intersects with a safety sphere around the quadrotor. We experimentally demonstrate that our method allows initiating evasive maneuvers early enough to avoid collisions. Abstract-We present a method to predict collisions with objects thrown at a quadrotor using a pair of dynamic vision sensors (DVS). Due to the micro-second temporal resolution of these sensors and the sparsity of their output, the object's trajectory can be estimated with minimal latency. Unlike standard cameras that send frames at a fixed frame rate, a DVS only transmits pixel-level brightness changes ("events") at the time they occur. Our method tracks spherical objects on the image plane using probabilistic trackers that are updated with each incoming event. The object's trajectory is estimated using an Extended Kalman Filter with a mixed state space that allows incorporation of both the object's dynamics and the measurement noise in the image plane. Using error-propagation techniques, we predict a collision if the 3σ-ellipsoid along the predicted trajectory intersects with a safety sphere around the quadrotor. We experimentally demonstrate that our method allows initiating evasive maneuvers early enough to avoid collisions.

“…tracked moving objects as clustered blob-like sources of events [11], [10]. The high-speed advantage of event cameras was also shown in [12] for a pencil-balancing robot. Tracking of the pencil was performed using a fast event-based Hough transform.…”

Section: A Event-based Feature Detection and Trackingmentioning

confidence: 86%

Low-latency visual odometry using event-based feature tracks

Kueng

2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Gallego

et al. 2016

154

159

New vision sensors, such as the Dynamic and Active-pixel Vision sensor (DAVIS), incorporate a conventional camera and an event-based sensor in the same pixel array. These sensors have great potential for robotics because they allow us to combine the benefits of conventional cameras with those of eventbased sensors: low latency, high temporal resolution, and high dynamic range. However, new algorithms are required to exploit the sensor characteristics and cope with its unconventional output, which consists of a stream of asynchronous brightness changes (called "events") and synchronous grayscale frames. In this paper, we present a lowlatency visual odometry algorithm for the DAVIS sensor using event-based feature tracks. Features are first detected in the grayscale frames and then tracked asynchronously using the stream of events. The features are then fed to an event-based visual odometry algorithm that tightly interleaves robust pose optimization and probabilistic mapping. We show that our method successfully tracks the 6-DOF motion of the sensor in natural scenes. This is the first work on event-based visual odometry with the DAVIS sensor using feature tracks. Low-Latency Visual Odometry using Event-based Feature Tracks Beat Kueng, Elias Mueggler, Guillermo Gallego and Davide ScaramuzzaAbstract-New vision sensors, such as the Dynamic and Active-pixel Vision sensor (DAVIS), incorporate a conventional camera and an event-based sensor in the same pixel array. These sensors have great potential for robotics because they allow us to combine the benefits of conventional cameras with those of event-based sensors: low latency, high temporal resolution, and high dynamic range. However, new algorithms are required to exploit the sensor characteristics and cope with its unconventional output, which consists of a stream of asynchronous brightness changes (called "events") and synchronous grayscale frames. In this paper, we present a lowlatency visual odometry algorithm for the DAVIS sensor using event-based feature tracks. Features are first detected in the grayscale frames and then tracked asynchronously using the stream of events. The features are then fed to an event-based visual odometry algorithm that tightly interleaves robust pose optimization and probabilistic mapping. We show that our method successfully tracks the 6-DOF motion of the sensor in natural scenes. This is the first work on event-based visual odometry with the DAVIS sensor using feature tracks. MULTIMEDIA MATERIALA video attachment to this work is available on the authors' webpage.