Spiking Neural Network Based Region Proposal Networks for Neuromorphic Vision Sensors

Acharya, Jyotibdha; Padala, Vandana Reddy; Basu, Arindam

doi:10.1109/iscas.2019.8702651

Cited by 4 publications

(15 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The major disadvantage of conventional SNNs is not differentiable, causing the popular training methods to be inapplicable. In the context of autonomous driving, a SNN architecture consisting of refractory, convolution, and clustering layers was presented [26]. It was designed with biorealistic LIF neurons and synapses.…”

Section: Snnsmentioning

confidence: 99%

Event-Based Neuromorphic Vision for Autonomous Driving: A Paradigm Shift for Bio-Inspired Visual Sensing and Perception

Chen

Cao

Conradt

et al. 2020

IEEE Signal Process. Mag.

172

View full text Add to dashboard Cite

s a bio-inspired and emerging sensor, an event-based neuromorphic vision sensor has a different working principle compared to the standard frame-based cameras, which leads to promising properties of low energy consumption, low latency, high dynamic range (HDR), and high temporal resolution. It poses a paradigm shift to sense and perceive the environment by capturing local pixel-level light intensity changes and producing asynchronous event streams. Advanced technologies for the visual sensing system of autonomous vehicles from standard computer vision to event-based neuromorphic vision have been developed. In this tutorial-like article, a comprehensive review of the emerging technology is given. First, the course of the development of the neuromorphic vision sensor that is derived from the understanding of biological retina is introduced. The signal processing techniques for event noise processing and event data representation are then discussed. Next, the signal processing algorithms and applications for event-based neuromorphic vision in autonomous driving and various assistance systems are reviewed. Finally, challenges and future research directions are pointed out. It is expected that this article will serve as a starting point for new researchers and engineers in the autonomous driving field and provide a bird's-eye view to both neuromorphic vision and autonomous driving research communities.

show abstract

Section: Snnsmentioning

confidence: 99%

Event-Based Neuromorphic Vision for Autonomous Driving: A Paradigm Shift for Bio-Inspired Visual Sensing and Perception

Chen

Cao

Conradt

et al. 2020

IEEE Signal Process. Mag.

172

View full text Add to dashboard Cite

show abstract

“…Spiking neural networks (SNNs) [45] have been applied to various event-based fields, including low-level tasks such as optical flow estimation [46][47][48], high-level tasks such as object recognition [49,50] and classification [51], and tasks concerning the 3D structure of the scene [52,53] and robotic visual perception [54]. Benosman et al [16] used a spiking neural network that is theoretically similar to the classical Lucas-Kanade algorithm to estimate visual motion, exploiting the sparse high temporal resolution event data.…”

Section: Data-driven Approachesmentioning

confidence: 99%

“…Tang et al [50] proposed a hierarchical feedforward spiking neural network for the classification of digital characters recorded by DVS. In addition, Acharya et al [51] presented a three-layer SNNbased region proposal network operating on event data and applied it to real recordings. Although SNNs have mainly been applied to classification problems [50,51,57], a recent research [58] unlocked the potential of SNNs to tackle numeric regression problems in the continuous-time domain for event-based data.…”

Section: Data-driven Approachesmentioning

confidence: 99%

“…In addition, Acharya et al [51] presented a three-layer SNNbased region proposal network operating on event data and applied it to real recordings. Although SNNs have mainly been applied to classification problems [50,51,57], a recent research [58] unlocked the potential of SNNs to tackle numeric regression problems in the continuous-time domain for event-based data. Benosman et al [52,53] solved the stereo-correspondence problem and estimated depth in a 3D scene in event-based vision with a spiking neural network working on neural processing devices.…”

Section: Data-driven Approachesmentioning

confidence: 99%

See 1 more Smart Citation

Data-Driven Technology in Event-Based Vision

Sun

Shi²,

Zhang³

et al. 2021

Complexity

View full text Add to dashboard Cite

Event cameras which transmit per-pixel intensity changes have emerged as a promising candidate in applications such as consumer electronics, industrial automation, and autonomous vehicles, owing to their efficiency and robustness. To maintain these inherent advantages, the trade-off between efficiency and accuracy stands as a priority in event-based algorithms. Thanks to the preponderance of deep learning techniques and the compatibility between bio-inspired spiking neural networks and event-based sensors, data-driven approaches have become a hot spot, which along with the dedicated hardware and datasets constitute an emerging field named event-based data-driven technology. Focusing on data-driven technology in event-based vision, this paper first explicates the operating principle, advantages, and intrinsic nature of event cameras, as well as background knowledge in event-based vision, presenting an overview of this research field. Then, we explain why event-based data-driven technology becomes a research focus, including reasons for the rise of event-based vision and the superiority of data-driven approaches over other event-based algorithms. Current status and future trends of event-based data-driven technology are presented successively in terms of hardware, datasets, and algorithms, providing guidance for future research. Generally, this paper reveals the great prospects of event-based data-driven technology and presents a comprehensive overview of this field, aiming at a more efficient and bio-inspired visual system to extract visual features from the external environment.

show abstract

“…While being great pieces of research, we feel that this is fundamentally not a good application for SNN since the original input signal is static and does not change with time. Instead, it might be more natural to use SNN as dynamical systems to track moving objects in video streams [18], [19] or classify signals that vary over time such as speech [20]. With this in mind, we propose the following desired characteristics for neuromorphic benchmarks:…”

Section: Neuromorphic Benchmarksmentioning

confidence: 99%

Is my Neural Network Neuromorphic? Taxonomy, Recent Trends and Future Directions in Neuromorphic Engineering

Bose

Acharya

Basu

2019

2019 53rd Asilomar Conference on Signals, Systems, and Computers

Self Cite

View full text Add to dashboard Cite

In this paper, we review recent work published over the last 3 years under the umbrella of Neuromorphic engineering to analyze what are the common features among such systems. We see that there is no clear consensus but each system has one or more of the following features:(1) Analog computing (2) Non von-Neumann Architecture and low-precision digital processing (3) Spiking Neural Networks (SNN) with components closely related to biology. We compare recent machine learning accelerator chips to show that indeed analog processing and reduced bit precision architectures have best throughput, energy and area efficiencies. However, pure digital architectures can also achieve quite high efficiencies by just adopting a non von-Neumann architecture. Given the design automation tools for digital hardware design, it raises a question on the likelihood of adoption of analog processing in the near future for industrial designs. Next, we argue about the importance of defining standards and choosing proper benchmarks for the progress of neuromorphic system designs and propose some desired characteristics of such benchmarks. Finally, we show brain-machine interfaces as a potential task that fulfils all the criteria of such benchmarks.

show abstract

Spiking Neural Network Based Region Proposal Networks for Neuromorphic Vision Sensors

Cited by 4 publications

References 17 publications

Event-Based Neuromorphic Vision for Autonomous Driving: A Paradigm Shift for Bio-Inspired Visual Sensing and Perception

Event-Based Neuromorphic Vision for Autonomous Driving: A Paradigm Shift for Bio-Inspired Visual Sensing and Perception

Data-Driven Technology in Event-Based Vision

Is my Neural Network Neuromorphic? Taxonomy, Recent Trends and Future Directions in Neuromorphic Engineering

Contact Info

Product

Resources

About