Towards Dynamic Monocular Visual Odometry Based on an Event Camera and IMU Sensor

Mohamed, Sherif Abdelmonem Sayed; Haghbayan, Mohammad-Hashem; Rabah, Mohammed; Heikkonen, Jukka; Tenhunen, Hannu; Plosila, Juha

doi:10.1007/978-3-030-38822-5_17

Cited by 9 publications

(7 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [14], the authors proposed another approach to generate event-frames by accumulating a fixed number of events, (e.g., 2000 events) to form a frame. In [8], the authors presented a dynamic slicing method to generate event-frames based on the velocity of the event camera and the number of edges in the environment, i.e., entropy. The main disadvantage of using such techniques is that they omit the innate asynchronous nature of the event cameras.…”

Section: Related Workmentioning

confidence: 99%

“…Execution time for Harris does not change significantly since the algorithm processes all pixels in the image and all captured images have the same size. On the other hand, event-based units, such as matching unit and lifetime calculation depends on the number of generated events that is based on the speed of the camera and the amount of information in the scene [8]. The execution time of both units is very low and especially the matching unit which has an average execution time of 3.6 microseconds.…”

Section: Experimental Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Asynchronous Corner Tracking Algorithm Based on Lifetime of Events for DAVIS Cameras

Mohamed

Haghbayan

Miele

et al. 2020

Advances in Visual Computing

Self Cite

View full text Add to dashboard Cite

Event cameras, i.e., the Dynamic and Active-pixel Vision Sensor (DAVIS) ones, capture the intensity changes in the scene and generates a stream of events in an asynchronous fashion. The output rate of such cameras can reach up to 10 million events per second in high dynamic environments. DAVIS cameras use novel vision sensors that mimic human eyes. Their attractive attributes, such as high output rate, High Dynamic Range (HDR), and high pixel bandwidth, make them an ideal solution for applications that require high-frequency tracking. Moreover, applications that operate in challenging lighting scenarios can exploit from the high HDR of event cameras, i.e., 140 dB compared to 60 dB of traditional cameras. In this paper, a novel asynchronous corner tracking method is proposed that uses both events and intensity images captured by a DAVIS camera. The Harris algorithm is used to extract features, i.e., frame-corners from keyframes, i.e., intensity images. Afterward, a matching algorithm is used to extract event-corners from the stream of events. Events are solely used to perform asynchronous tracking until the next keyframe is captured. Neighboring events, within a window size of 5x5 pixels around the event-corner, are used to calculate the velocity and direction of extracted event-corners by fitting the 2D planar using a randomized Hough transform algorithm. Experimental evaluation showed that our approach is able to update the location of the extracted corners up to 100 times during the blind time of traditional cameras, i.e., between two consecutive intensity images.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Experimental Evaluationmentioning

confidence: 99%

Asynchronous Corner Tracking Algorithm Based on Lifetime of Events for DAVIS Cameras

Mohamed

Haghbayan

Miele

et al. 2020

Advances in Visual Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…For instance, in [15] the temporal window is based on the lifetime of the event. In [16], authors presented an algorithm to set the temporal window dynamically based on the amount of the information in the scene, i.e., entropy and the camera motion. In general, such methods present various drawbacks based on the complexity of the environment and the motion speed, such as generating blurry event-frames due to overaccumulating events or generating noisy frames by underaccumulating the sufficient amount of events to reconstruct the scene.…”

Section: A Intensity-based Corner Detectionmentioning

confidence: 99%

Dynamic Resource-aware Corner Detection for Bio-inspired Vision Sensors

Mohamed¹,

Haghbayan²,

Miele³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Event-based cameras are vision devices that transmit only brightness changes with low latency and ultra-low power consumption. Such characteristics make event-based cameras attractive in the field of localization and object tracking in resource-constrained systems. Since the number of generated events in such cameras is huge, the selection and filtering of the incoming events are beneficial from both increasing the accuracy of the features and reducing the computational load. In this paper, we present an algorithm to detect asynchronous corners form a stream of events in real-time on embedded systems. The algorithm is called the Three Layer Filtering-Harris or TLF-Harris algorithm. The algorithm is based on an events' filtering strategy whose purpose is 1) to increase the accuracy by deliberately eliminating some incoming events, i.e., noise and 2) to improve the real-time performance of the system, i.e., preserving a constant throughput in terms of input events per second, by discarding unnecessary events with a limited accuracy loss. An approximation of the Harris algorithm, in turn, is used to exploit its high-quality detection capability with a low-complexity implementation to enable seamless real-time performance on embedded computing platforms. The proposed algorithm is capable of selecting the best corner candidate among neighbors and achieves an average execution time savings of 59% compared with the conventional Harris score. Moreover, our approach outperforms the competing methods, such as eFAST, eHarris, and FA-Harris, in terms of real-time performance, and surpasses Arc* in terms of accuracy.

show abstract

“…Another example is a mobile robot with an event-based camera as the sensor. Here, both the environmental complexity and the robot speed affect the number of generated events at each instance of time and consequently the event processing workload [15], [16].…”

Section: Introductionmentioning

confidence: 99%

Energy-Efficient Mobile Robot Control via Run-time Monitoring of Environmental Complexity and Computing Workload

Mohamed¹,

Haghbayan²,

Miele³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We propose an energy-efficient controller to minimize the energy consumption of a mobile robot by dynamically manipulating the mechanical and computational actuators of the robot. The mobile robot performs real-time vision-based applications based on an event-based camera. The actuators of the controller are CPU voltage/frequency for the computation part and motor voltage for the mechanical part. We show that independently considering speed control of the robot and voltage/frequency control of the CPU does not necessarily result in an energy-efficient solution. In fact, to obtain the highest efficiency, the computation and mechanical parts should be controlled together in synergy. We propose a fast hill-climbing optimization algorithm to allow the controller to find the best CPU/motor configuration at run-time and whenever the mobile robot is facing a new environment during its travel. Experimental results on a robot with Brushless DC Motors, Jetson TX2 board as the computing unit, and a DAVIS-346 event-based camera show that the proposed control algorithm can save battery energy by an average of 50.5%, 41%, and 30%, in low-complexity, medium-complexity, and high-complexity environments, over baselines.

show abstract

Towards Dynamic Monocular Visual Odometry Based on an Event Camera and IMU Sensor

Cited by 9 publications

References 26 publications

Asynchronous Corner Tracking Algorithm Based on Lifetime of Events for DAVIS Cameras

Asynchronous Corner Tracking Algorithm Based on Lifetime of Events for DAVIS Cameras

Dynamic Resource-aware Corner Detection for Bio-inspired Vision Sensors

Energy-Efficient Mobile Robot Control via Run-time Monitoring of Environmental Complexity and Computing Workload

Contact Info

Product

Resources

About