Embedded Vision System for Real-Time Object Tracking using an Asynchronous Transient Vision Sensor

Litzenberger, M.; Posch, C.; Bauer, Dietmar; Belbachir, Ahmed Nabil; Schön, Peter; Kohn, Bernhard; Garn, Heinrich

doi:10.1109/dspws.2006.265448

Cited by 124 publications

(106 citation statements)

References 10 publications

Supporting

Mentioning

106

Contrasting

Order By: Relevance

“…[38][39][40][41]) and high speed robotics [42,43]. Eight papers on event based sensing have been accepted to the very competitive IEEE International Solid State Circuits Conference since 2003 (two on auditory sensors [44,45] and six on vision sensors [9,16,27,33,46,47]), showing that this approach is starting to impact mainstream electronics.…”

Section: Resultsmentioning

confidence: 99%

Activity-driven, event-based vision sensors

Delbrück

Linares-Barranco

Culurciello

et al. 2010

Proceedings of 2010 IEEE International Symposium on Circuits and Systems

188

109

View full text Add to dashboard Cite

The four chips [1][2][3][4] presented in the special session on "Activity-driven, event-based vision sensors" quickly output compressed digital data in the form of events. These sensors reduce redundancy and latency and increase dynamic range compared with conventional imagers. The digital sensor output is easily interfaced to conventional digital post processing, where it reduces the latency and cost of post processing compared to imagers. The asynchronous data could spawn a new area of DSP that breaks from conventional Nyquist rate signal processing. This paper reviews the rationale and history of this event-based approach, introduces sensor functionalities, and gives an overview of the papers in this session. The paper concludes with a brief discussion on open questions.

show abstract

Section: Resultsmentioning

confidence: 99%

Activity-driven, event-based vision sensors

Delbrück

Linares-Barranco

Culurciello

et al. 2010

Proceedings of 2010 IEEE International Symposium on Circuits and Systems

188

109

View full text Add to dashboard Cite

show abstract

“…Since they assume that the background is constant and the system reacts only to changes, their system cannot be applied in a dynamic environment. The same goes for the work of Litzenbeger et al [22] which used this method to track pedestrians and vehicles.…”

Section: State Of the Artmentioning

confidence: 81%

A mechanism to improve Stereo Vision Systems in automated heterogeneous platoons

Alfraheed¹,

Droge²,

Klingender³

et al. 2012

2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC)

View full text Add to dashboard Cite

Due to their low price and good quality, Stereo Vision Systems (SVS) are recently considered as a key factor to gather actual information about the object of interest. Today, automated highway systems (AHS) for urban and highway environment were developed without the use of a stereo vision system. In future, the application of AHS should be extended to unstructured environments (e.g. desert) and be adapted to heterogeneous vehicles. In this context, the stereo vision system could enable the platoon to be independent from environmental structure (e.g. lane markings) through its ability to detect, track, locate and recognize heterogeneous vehicles. So far, the need for high accuracy prevents SVS to be applied in automated heterogeneous platoon. In this paper a mechanism towards this is presented, where some behavioral properties have to be satisfied in terms of unstructured environment and heterogeneous platoons. Within a heterogeneous platoon, the back view of a preceding vehicle (BVPV) is considered as a reference point for the lateral and longitudinal control. The key idea of the proposed mechanism is to confirm that the distance of the BVPV can be extracted without depending on the movement of the preceding vehicle. Furthermore, the proposed mechanism has to ensure that features extracted from the back view are suitable to implement successfully the calibration process at around 10m distance. With the proposed SVS mechanism some of behavioral properties have to be satisfied in terms of unstructured environment and AHS. These properties are reliability, performance and robustness. Compared to other methods which use a SVS, the proposed mechanism distinguishes itself through adapting to dynamic environment and extracting the necessary features for the calibration process.

show abstract

“…Thousands of events are triggered in the time between two frames since, due to the sensor's motion, intensity changes occur at all pixels. tracked moving objects as clustered blob-like sources of events [11], [10]. The high-speed advantage of event cameras was also shown in [12] for a pencil-balancing robot.…”

Section: A Event-based Feature Detection and Trackingmentioning

confidence: 87%

Low-latency visual odometry using event-based feature tracks

Kueng

Mueggler

Gallego

et al. 2016

2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

142

159

View full text Add to dashboard Cite

New vision sensors, such as the Dynamic and Active-pixel Vision sensor (DAVIS), incorporate a conventional camera and an event-based sensor in the same pixel array. These sensors have great potential for robotics because they allow us to combine the benefits of conventional cameras with those of eventbased sensors: low latency, high temporal resolution, and high dynamic range. However, new algorithms are required to exploit the sensor characteristics and cope with its unconventional output, which consists of a stream of asynchronous brightness changes (called "events") and synchronous grayscale frames. In this paper, we present a lowlatency visual odometry algorithm for the DAVIS sensor using event-based feature tracks. Features are first detected in the grayscale frames and then tracked asynchronously using the stream of events. The features are then fed to an event-based visual odometry algorithm that tightly interleaves robust pose optimization and probabilistic mapping. We show that our method successfully tracks the 6-DOF motion of the sensor in natural scenes. This is the first work on event-based visual odometry with the DAVIS sensor using feature tracks. Low-Latency Visual Odometry using Event-based Feature Tracks Beat Kueng, Elias Mueggler, Guillermo Gallego and Davide ScaramuzzaAbstract-New vision sensors, such as the Dynamic and Active-pixel Vision sensor (DAVIS), incorporate a conventional camera and an event-based sensor in the same pixel array. These sensors have great potential for robotics because they allow us to combine the benefits of conventional cameras with those of event-based sensors: low latency, high temporal resolution, and high dynamic range. However, new algorithms are required to exploit the sensor characteristics and cope with its unconventional output, which consists of a stream of asynchronous brightness changes (called "events") and synchronous grayscale frames. In this paper, we present a lowlatency visual odometry algorithm for the DAVIS sensor using event-based feature tracks. Features are first detected in the grayscale frames and then tracked asynchronously using the stream of events. The features are then fed to an event-based visual odometry algorithm that tightly interleaves robust pose optimization and probabilistic mapping. We show that our method successfully tracks the 6-DOF motion of the sensor in natural scenes. This is the first work on event-based visual odometry with the DAVIS sensor using feature tracks. MULTIMEDIA MATERIALA video attachment to this work is available on the authors' webpage.

show abstract

Embedded Vision System for Real-Time Object Tracking using an Asynchronous Transient Vision Sensor

Cited by 124 publications

References 10 publications

Activity-driven, event-based vision sensors

Activity-driven, event-based vision sensors

A mechanism to improve Stereo Vision Systems in automated heterogeneous platoons

Low-latency visual odometry using event-based feature tracks

Contact Info

Product

Resources

About