SeqXFilter: A Memory-efficient Denoising Filter for Dynamic Vision Sensors

Abstract

Neuromorphic event-based dynamic vision sensors (DVS) have much faster sampling rates and a higher dynamic range than frame-based imaging sensors. However, they are sensitive to background activity (BA) events that are unwanted. There are some filters for tackling this problem based on spatio-temporal correlation. However, they are either memory-intensive or computing-intensive. We propose SeqXFilter, a spatio-temporal correlation filter with only a past event window that has an O(1) space complexity and has simple computations. We explore the spatial correlation of an event with its past few events by analyzing the distribution of the events when applying different functions on the spatial distances. We find the best function to check the spatio-temporal correlation for an event for SeqXFilter, best separating real events and noise events. We not only give the visual denoising effect of the filter but also use two metrics for quantitatively analyzing the filter's performance. Four neuromorphic event-based datasets, recorded from four DVS with different output sizes, are used for validation of our method. The experimental results show that SeqXFilter achieves similar performance as baseline NNb filters, but with extremely small memory cost and simple computation logic. IntroductionResearch on neuromorphic event-based sensors ("silicon retinae") started a few decades back [1]. Recently, the technology has matured to a stage where there have been some commercially available sensors. Some of the popular sensors are Dynamic Vision Sensor (DVS) [2], Asynchronous Timebased Image Sensor (ATIS) [3], the sensitive DVS [4], the Dynamic and Active pixel Vision Sensor (DAVIS) [5] and the Celex-IV [6]. Different from conventional frame-based imaging sensors that work by sampling the scene at a fixed temporal rate (typically 30 frames per second [7]), these neuromorphic sensors detect dynamic changes in illumination. This results in a higher dynamic range, higher sampling rate, and lower power consumption. They have been utilized in applications such as pose estimation [8], gesture-based remote control [9], corner detection [10], and drones [11].However, these sensors will produce background activity (BA) events under constant illumination, which are caused by temporal noise and junction leakage currents [2,12,13]. There are already multiple noise filtering methods for event-based data available.The most commonly employed method is the Nearest Neighbor (NNb) filter based on spatio-temporal correlation [12,14,15]. However, these spatio-temporal NNb filters are memory intensive. For a DVS with N×N pixels, the minimum space complexity is at O(N) level, which puts a burden on the limited programmable logic (PL) embedded in the sensor head. There are two main reasons for the high space complexity of the current spatio-temporal filters. First, the memory size of these filters are dependent on the DVS output size N for the convenience of detecting spatio-temporal Preprint. Under review.