UAV Detection: A STDP Trained Deep Convolutional Spiking Neural Network Retina-Neuromorphic Approach

Abstract

The Dynamic Vision Sensor (DVS) has many attributes, such as sub-millisecond response time along with a good low light dy-namic range, that allows it to be well suited to the task for UAV De-tection. This paper proposes a system that exploits the features of an event camera solely for UAV detection while combining it with a Spik-ing Neural Network (SNN) trained using the unsupervised approach of Spike Time-Dependent Plasticity (STDP), to create an asynchronous, low power system with low computational overhead. Utilising the unique features of both the sensor and the network, this result in a system that is robust to a wide variety in lighting conditions, has a high temporal resolution, propagates only the minimal amount of information through the network, while training using the equivalent of 43,000 images. The network returns a 92% detection rate when shown other objects and can detect a UAV with less than 1% of pixels on the sensor being used for processing.

Perception Understanding Action: Adding Understanding to the Perception Action Cycle With Spiking Segmentation

“…The adaptive neuron thresholding used within this paper builds upon the Pre-Emptive Neuron Thresholding (Kirkland et al, 2019(Kirkland et al, , 2020. Improvements are made by no longer solely relying on synaptic scaling from the input number of spikes as a means of homoeostasis.…”

SpikeSEG: Spiking Segmentation via STDP Saliency Mapping

“…A progression of the Pre-Emptive Neuron Thresholding (PENT) processes described in [39], with the adaptation now being able to affect all encoding Conv layers within the network. The thresholding is based on the homoeostasis mechanism called synaptic scaling [40], normally taking effect after hours or even days of high neuronal activity, to try and reduce activity.…”

Neuromorphic technologies for defence and security

“…Simple methods for detection and classification were shown in Kheradpisheh et al, 49 and then utilised on UAVs within the following paper by Kirkand et al 50 The latter proposes a low latency method of detection, that could successfully detect UAVs in a number of scenarios including adverse lighting conditions. The paper also highlights that the method can work with data direct from a NM vision sensor (in this case the DVS 240) and with video converted into a spiking format, using a difference of Gaussian's filter to convert the frames of the video into a temporal spike pattern based on intensity.…”

“…Semantic segmentation of spiking based images through the use of a spiking fully convolutional encoding decoding network 51 can be seen as the next progressive step of a detection based system. 49,50 This novel use of an encoderdecoder structure with learned weights from an unsupervised STDP approach allows low latency asynchronous output of semantically segmented images. Within a semantic segmentation network, the learned features of the STDP approach similar to that seen in the previous case study are now used in post classification.…”