Asynchronous event-based clustering and tracking for intrusion monitoring in UAS

Abstract

Automatic surveillance and monitoring using Unmanned Aerial Systems (UAS) require the development of perception systems that robustly work under different illumination conditions. Event cameras are neuromorphic sensors that capture the illumination changes in the scene with very low latency and high dynamic range. Although recent advances in eventbased vision have explored the use of event cameras onboard UAS, most techniques group events in frames and, therefore, do not fully exploit the sequential and asynchronous nature of the event stream. This paper proposes a fully asynchronous scheme for intruder monitoring using UAS. It employs efficient event clustering and feature tracking modules and includes a sampling mechanism to cope with the computational cost of event-by-event processing adapting to on-board hardware computational constraints. The proposed scheme was tested on a real multirotor in challenging scenarios showing significant accuracy and robustness to lighting conditions.

An Outline of Multi-Sensor Fusion Methods for Mobile Agents Indoor Navigation

“…In addition the discussed datasets above, Table 3 presents a view for various multi-sensors agent navigation datasets, including UZH-FPV Drone Racing [ 125 ], TUM RGB-D Dataset [ 126 ], ScanNet [ 127 ], NYU V2 [ 128 ], InteriorNet [ 129 ], SceneNet RGB-D [ 130 ], and others [ 131 , 132 , 133 , 134 , 135 , 136 , 137 , 138 , 139 , 140 , 141 , 142 , 143 , 144 ], etc. These datasets provide the basic requirements of simulation and evaluation of multi-sensor fusion in experiments.…”

The GRIFFIN Perception Dataset: Bridging the Gap Between Flapping-Wing Flight and Robotic Perception

“…such as visual servoing [19], motion segmentation [20], surveillance tasks [21], robot localization [22], and onboard computation load management [23], among many others. A number of datasets for event-based vision have been presented exploring the advantages of event cameras onboard aerial robots [13], [14], [21], [24]. The works in [21] and [24] provide sequences recorded onboard multirotors used to evaluate event-based methods for tracking moving objects.…”

Asynchronous Event-based Line Tracking for Time-to-Contact Maneuvers in UAS

“…The advantages of event cameras have motivated increasing research interest of the robotics and computer vision communities [3] [17]. A wide variety of methods have been used for feature extraction [18], clustering [19], tracking [20], optical flow computation [21], detect objects in motion [22], and SLAM [23], among many others. A full review of event processing methods can be found in [3].…”

“…To ensure the statistical consistency of the EKF, event-by-event processing and event-image processing use different events. The events received from the camera are randomly sampled as in [22]. γ e and γ i are the percentages of the input events that are sent to the event-by-event and the image-event processing modules, respectively.…”