Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision

Kugele, Alexander; Pfeil, Thomas; Pfeiffer, Michael; Chicca, Elisabetta

doi:10.1007/978-3-030-92659-5_19

Cited by 23 publications

(22 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A large amount of research has been done presenting methods that increase the power efficiency of ML algorithms running on embedded systems, such as event-based inference methods that induce a “smart triggering” of the power-hungry inference algorithm under certain criteria only. In the machine-vision application presented in [ 7 ], the authors used event-based vision sensors that encode scenes using streams of events that represent local pixel-wise brightness changes, rather than full image frames. This resulted in a sparse, energy-efficient representation of the scene, as well as low inference latency.…”

Section: Related Workmentioning

confidence: 99%

Power Efficient Machine Learning Models Deployment on Edge IoT Devices

Fanariotis

Orphanoudakis

Kotrotsios

et al. 2023

Sensors

View full text Add to dashboard Cite

Computing has undergone a significant transformation over the past two decades, shifting from a machine-based approach to a human-centric, virtually invisible service known as ubiquitous or pervasive computing. This change has been achieved by incorporating small embedded devices into a larger computational system, connected through networking and referred to as edge devices. When these devices are also connected to the Internet, they are generally named Internet-of-Thing (IoT) devices. Developing Machine Learning (ML) algorithms on these types of devices allows them to provide Artificial Intelligence (AI) inference functions such as computer vision, pattern recognition, etc. However, this capability is severely limited by the device’s resource scarcity. Embedded devices have limited computational and power resources available while they must maintain a high degree of autonomy. While there are several published studies that address the computational weakness of these small systems-mostly through optimization and compression of neural networks- they often neglect the power consumption and efficiency implications of these techniques. This study presents power efficiency experimental results from the application of well-known and proven optimization methods using a set of well-known ML models. The results are presented in a meaningful manner considering the “real world” functionality of devices and the provided results are compared with the basic “idle” power consumption of each of the selected systems. Two different systems with completely different architectures and capabilities were used providing us with results that led to interesting conclusions related to the power efficiency of each architecture.

show abstract

Section: Related Workmentioning

confidence: 99%

Power Efficient Machine Learning Models Deployment on Edge IoT Devices

Fanariotis

Orphanoudakis

Kotrotsios

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…This demonstrates that learning to detect objects from events is possible, but it still requires deep neural networks that do not take full advantage of the properties of event-data and that would be difficult to embed in power constrained environment. Hybrid-SNN [21] proposes a partial solution by presenting an hybrid neural network composed of a SNN backbone for efficient event-based feature extraction, and an ANN head to solve object detection tasks. To the best of our knowledge, the work presented in our paper is the first complete spiking neural network capable of doing object detection.…”

Section: Object Detection On Event Datamentioning

confidence: 99%

Object Detection with Spiking Neural Networks on Automotive Event Data

Cordone¹,

Miramond²,

Thierion³

2022

Preprint

View full text Add to dashboard Cite

Automotive embedded algorithms have very high constraints in terms of latency, accuracy and power consumption. In this work, we propose to train spiking neural networks (SNNs) directly on data coming from event cameras to design fast and efficient automotive embedded applications. Indeed, SNNs are more biologically realistic neural networks where neurons communicate using discrete and asynchronous spikes, a naturally energy-efficient and hardware friendly operating mode. Event data, which are binary and sparse in space and time, are therefore the ideal input for spiking neural networks. But to date, their performance was insufficient for automotive real-world problems, such as detecting complex objects in an uncontrolled environment. To address this issue, we took advantage of the latest advancements in matter of spike backpropagation -surrogate gradient learning, parametric LIF, SpikingJelly frameworkand of our new voxel cube event encoding to train 4 different SNNs based on popular deep learning networks: SqueezeNet, VGG, MobileNet, and DenseNet. As a result, we managed to increase the size and the complexity of SNNs usually considered in the literature. In this paper, we conducted experiments on two automotive event datasets, establishing new state-of-the-art classification results for spiking neural networks. Based on these results, we combined our SNNs with SSD to propose the first spiking neural networks capable of performing object detection on the complex GEN1 Automotive Detection event dataset.

show abstract

“…There are many examples of converting data directly from sensors [19][20][21], intelligent systems with controlling manipulators [11] [22], and robots [23][24] [25]. Moreover, performing detection and recognition tasks [26][27] [28], and processing numerical data with Neural Engineering Framework (NEF) [29][30] [31] can also be done with SNNs.…”

Section: Spiking Neural Network (Snn)mentioning

confidence: 99%

“…This type of hybrid SNN structure shows some neurons having a feed-forward connection whereas others have recurrent connections. This type of hybrid structures are often used for end-to-end training of SNNs for tasks like object detection and pattern recognition [26]. Experiments with a hybrid approach demonstrated promising results with less computational cost [47].…”

Section: ) Hybrid Neural Network Structuresmentioning

confidence: 99%

See 1 more Smart Citation

Neuromorphic Computing for Interactive Robotics: A Systematic Review

2022

View full text Add to dashboard Cite

Modelling functionalities of the brain in human-robot interaction contexts requires a real-time understanding of how each part of a robot (motors, sensors, emotions, etc.) works and how they interact all together to accomplish complex behavioural tasks while interacting with the environment. Human brains are very efficient as they process the information using event-based impulses also known as spikes, which make living creatures very efficient and able to outperform current mainstream robotic systems in almost every task that requires real-time interaction. In recent years, combined efforts by neuroscientists, biologists, computer scientists and engineers make it possible to design biologically realistic hardware and models that can endow the robots with the required human-like processing capability based on neuromorphic computing and Spiking Neural Network (SNN). However, while some attempts have been made, a comprehensive combination of neuromorphic computing and robotics is still missing. In this article, we present a systematic review of neuromorphic computing applications for socially interactive robotics. We first introduce the basic principles, models and architectures of neuromorphic computation. The remaining articles are classified according to the applications they focus on. Finally, we identify the potential research topics for fully integrated socially interactive neuromorphic robots.

show abstract

Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision

Cited by 23 publications

References 38 publications

Power Efficient Machine Learning Models Deployment on Edge IoT Devices

Power Efficient Machine Learning Models Deployment on Edge IoT Devices

Object Detection with Spiking Neural Networks on Automotive Event Data

Neuromorphic Computing for Interactive Robotics: A Systematic Review

Contact Info

Product

Resources

About