Modeling-based design of brain-inspired spiking neural networks with RRAM learning synapses

Pedretti, Giacomo; Bianchi, Stefano; Milo, Valerio; Calderoni, A.; Ramaswamy, Nirmal; Ielmini, Daniele

doi:10.1109/iedm.2017.8268467

Cited by 21 publications

(21 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These feature maps are classified in the third block. Note that from class to class the pattern density P of the feature map, namely the number of responses equal to VDD with respect to the overall number of responses (Pedretti et al, 2017), can change. This results in an unfair competition between the feature maps presented to the WTA network, since the internal spiking threshold of every POST is initially fixed to a nominal value.…”

Section: Block 2: Combinational Logic For Pattern Equalizationmentioning

confidence: 99%

“…Non-trained classes can generate different equalized patterns due to different combinations of responses from the FFs. However, the generated feature maps have different probabilities of appearance R P , which could complicate the learning procedure (Pedretti et al, 2017). In fact, as shown in Figure 5B, the first pattern, which has a R P = 46%, achieves a good separation between pattern and background average conductance, while the second and the third ones (28% and 15%, respectively) show a smaller window.…”

Section: Block 3: Winner-take-all Network For Plastic Adaptationmentioning

confidence: 99%

“…STDP (Pedretti et al, 2017). Once this data has been correctly sent and stored, the MNIST images are transferred one by one.…”

Section: Figure 4 | (A)mentioning

confidence: 99%

See 2 more Smart Citations

Bio-Inspired Techniques in a Fully Digital Approach for Lifelong Learning

2020

View full text Add to dashboard Cite

Lifelong learning has deeply underpinned the resilience of biological organisms respect to a constantly changing environment. This flexibility has allowed the evolution of parallel-distributed systems able to merge past information with new stimulus for accurate and efficient brain-computation. Nowadays, there is a strong attempt to reproduce such intelligent systems in standard artificial neural networks (ANNs). However, despite some great results in specific tasks, ANNs still appear too rigid and static in real life respect to the biological systems. Thus, it is necessary to define a new neural paradigm capable of merging the lifelong resilience of biological organisms with the great accuracy of ANNs. Here, we present a digital implementation of a novel mixed supervised-unsupervised neural network capable of performing lifelong learning. The network uses a set of convolutional filters to extract features from the input images of the MNIST and the Fashion-MNIST training datasets. This information defines an original combination of responses of both trained classes and non-trained classes by transfer learning. The responses are then used in the subsequent unsupervised learning based on spike-timing dependent plasticity (STDP). This procedure allows the clustering of non-trained information thanks to bio-inspired algorithms such as neuronal redundancy and spike-frequency adaptation. We demonstrate the implementation of the neural network in a fully digital environment, such as the Xilinx Zynq-7000 System on Chip (SoC). We illustrate a user-friendly interface to test the network by choosing the number and the type of the non-trained classes, or drawing a custom pattern on a tablet. Finally, we propose a comparison of this work with networks based on memristive synaptic devices capable of continual learning, highlighting the main differences and capabilities respect to a fully digital approach.

show abstract

Section: Block 2: Combinational Logic For Pattern Equalizationmentioning

confidence: 99%

Section: Block 3: Winner-take-all Network For Plastic Adaptationmentioning

confidence: 99%

See 1 more Smart Citation

Bio-Inspired Techniques in a Fully Digital Approach for Lifelong Learning

2020

View full text Add to dashboard Cite

show abstract

“…Synapse depression thus occurs according to the STDP rule. However, since noise spiking density is relatively small compared to the pattern density, the depression rate is lower than the potentiation rate, the latter approaching the one-or few-shot learning speed [80]. Higher noise density leads to a faster background depression, hence increased overall training speed.…”

Section: Unsupervised Learning By Stdp Has Been Demonstrated In Hardwmentioning

confidence: 99%

Neuromorphic computing with resistive switching memory devices

Ielmini

Ambrogio

2019

Advances in Non-Volatile Memory and Storage Technology

View full text Add to dashboard Cite

Brain-inspired computing has been the subject of an intense research in the last decades, with the aim of recreating some of the cognitive computing functions of the human brain in silicon-based hardware. In this frame, resistive switching memories (RRAM) and other emerging memory technologies are extremely promising as they offer memory and plasticity with high scaling capability, thus enabling the integration of a high density of synapses and neurons. This chapter summarizes the status and challenges of RRAM-based neuromorphic engineering. RRAM synapses are described within the frame of various neural network architectures, such as artificial neural networks (ANNs) and spiking neural networks (SNNs). First, ANNs with deep neural network architectures are described in terms of their operation during inference and learning, referring to the typical backpropagation scheme for supervised training. The challenges for high-density, high-functionality ANNs for computer vision with RRAM synapses are addressed.RRAM circuits enabling spike-timing dependent plasticity (STDP) and their use for unsupervised learning in feed-forward and recurrent networks are then presented. A hardware SNN for unsupervised learning by STDP in RRAM synapses is illustrated, demonstrating learning of static and dynamic patterns with both binary and gray-scale values. Finally, an outlook on the prospects of RRAM for future cognitive computing is given.

show abstract

“…Both an analytical model and a Monte Carlo (MC) model were presented to explain experimental data from a neuromorphic hardware. It was shown that the MC model of RRAM circuits and the analytical compact model of the STDP dynamics accurately predicted the learning behaviors in a spiking network with RRAM synapses (Pedretti et al, 2017).…”

Section: Brain-inspired Computingmentioning

confidence: 99%

Progress and Challenges of Neuroscience and Brain-inspired Artificial Intelligence

Wang¹,

Alexander²

2019

Neuroscience International

View full text Add to dashboard Cite

Neuroscience and brain-inspired artificial intelligence are significant research areas. Many countries have launched brain-related projects in which neuroscience and brain-inspired artificial intelligence are major targeted areas to increase national interests and enhance their strength in key areas such as military and homeland security in the competitive global world. Methods, emerging technologies, and progress in neuroscience and brain-inspired artificial intelligence are introduced in this paper that specifically include brain-inspired computing, brain association graph, brain networks, the connectome, brain reconstruction, imaging technologies used for the brain, chips and devices inspired by the human brain, brain-computer interface or brain-machine interfaces, cyborg, neuro-robotics, and quantum robotics. Challenges in some of the topics are also presented and discussed.

show abstract

Modeling-based design of brain-inspired spiking neural networks with RRAM learning synapses

Cited by 21 publications

References 5 publications

Bio-Inspired Techniques in a Fully Digital Approach for Lifelong Learning

Bio-Inspired Techniques in a Fully Digital Approach for Lifelong Learning

Neuromorphic computing with resistive switching memory devices

Progress and Challenges of Neuroscience and Brain-inspired Artificial Intelligence

Contact Info

Product

Resources

About