Event - Driven Tactile Learning with Location Spiking Neurons

Abstract

The sense of touch is essential for a variety of daily tasks. New advances in event-based tactile sensors and Spiking Neural Networks (SNNs) spur the research in event-driven tactile learning. However, SNN-enabled event-driven tactile learning is still in its infancy due to the limited representative abilities of existing spiking neurons and high spatio-temporal complexity in the data. In this paper, to improve the representative capabilities of existing spiking neurons, we propose a novel neuron model called "location spiking neuron", which enables us to extract features of event-based data in a novel way. Moreover, based on the classical Time Spike Response Model (TSRM), we develop a specific location spiking neuron model -Location Spike Response Model (LSRM) that serves as a new building block of SNNs 1 . Furthermore, we propose a hybrid model which combines an SNN with TSRM neurons and an SNN with LSRM neurons to capture the complex spatio-temporal dependencies in the data. Extensive experiments demonstrate the significant improvements of our models over other works on event-driven tactile learning and show the superior energy efficiency of our models and location spiking neurons, which may unlock their potential on neuromorphic hardware. 2

Boost event-driven tactile learning with location spiking neurons

“…We believe such neuron models can have a broad impact on the SNN community and spur the research on spike-based learning. Extensive experiments on real-world datasets show that the Hybrid_LIF_GNN significantly outperforms the state-ofthe-art methods for event-driven tactile learning, including the Hybrid_SRM_FC (Kang et al, 2022). Moreover, the computational cost evaluation demonstrates the highefficiency benefits of the Hybrid_LIF_GNN and LLIF neurons, which may unlock their potential on neuromorphic hardware.…”

“…Portions of this work "Event-Driven Tactile Learning with Location Spiking Neurons (Kang et al, 2022)" were accepted by IJCNN 2022 and an oral presentation was given at the IEEE WCCI 2022. In the conference paper, we proposed location spiking neurons and demonstrated the dynamics of LSRM neurons.…”

“…Portions of this work Event-Driven Tactile Learning with Location Spiking Neurons (Kang et al, 2022) were accepted by IJCNN 2022 and orally presented at the IEEE WCCI in 2022.…”

Recent Advances and Future Prospects for Memristive Materials, Devices, and Systems

“…A second network layer was then used to analyze the excited reservoir state (formed by the collective memristor states) and perform functions such as classification and signal reconstruction. , The ability to directly process information at the device level leaves plenty of room for imagination in how a system can be built to effectively utilize these devices. One vision is to implement a system comprising second-order memristors and STM memristors in a hierarchical manner, inspired by the columnar structure in the neocortex. − The system can be directly integrated with sensors that produce spatiotemporal data, such as neural probes, touch sensors, or an event-based camera. − The memristors can directly receive the input spike trains from the sensors, and natively process spatiotemporal information hidden in the spike trains for tasks such as object detection or motor movement control . Compared with conventional approaches that need to store and accumulate the sensor inputs, directly processing the streaming spikes can lead to faster response, lower power, and more robust noise tolerance.…”

Feasibility study on the application of a spiking neural network in myoelectric control systems

“…SNN is event-driven and can be combined with event-based sensors to provide an efficient bionic solution for pattern recognition tasks. Specifically, for tactile object recognition based on event-based tactile sensors (Taunyazov et al, 2020), Kang et al proposed a location spiking neuron based on time-dependent spiking neurons and constructed a hybrid model using both neurons, verifying that the model can better capture the complex spatio-temporal dependencies in eventdriven tactile data (Kang et al, 2023). For gesture recognition tasks based on event-based dynamic vision sensors (DVSs) (Brandli et al, 2014), Xing et al proposed a new spiking convolutional recurrent neural network (SCRNN) architecture, which used convolutional operations and recursive connectivity to maintain spatial and temporal relationships in event-based sequential data and achieved 96.59% accuracy in 10-class gesture recognition and 90.28% accuracy in 11-class gesture recognition (Xing et al, 2020).…”