Demonstration of neuromorphic sequence learning on a memristive array

Siegel, Sebastian; Ziegler, Tobias; Bouhadjar, Younes; Tetzlaff, Tom; Waser, Rainer; Dittmann, Ralf W.; Wouters, Dirk J.

doi:10.1145/3584954.3585000

Cited by 3 publications

(3 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Instead, the algorithm-hardware co-design is strictly oriented on the principles of compute-in-memory and realizes inference and update plasticity steps without the need of a dedicated memory controller, only with the peripheral neuron circuitry. Recently, the functionality of the MemSpikingTM algorithm was even demonstrated for a small task size on a real memristive crossbar array (Siegel et al 2023). With this learning rule we are able to train even complex high-order sequences to the network, showing how connection sparsity evolves during training and benefits the energy efficiency.…”

Section: Discussionmentioning

confidence: 92%

System model of neuromorphic sequence learning on a memristive crossbar array

Siegel

Bouhadjar

Tetzlaff

et al. 2023

Neuromorph. Comput. Eng.

Self Cite

View full text Add to dashboard Cite

Machine learning models for sequence learning and processing often suffer from high energy consumption and require large amounts of training data. The brain presents more efficient solutions to how these types of tasks can be solved. While this has inspired the conception of novel brain-inspired algorithms, their realizations remain constrained to conventional von-Neumann machines. Therefore, the potential power efficiency of the algorithm cannot be exploited due to the inherent memory bottleneck of the computing architecture.  Therefore, we present in this paper a dedicated hardware implementation of a biologically plausible version of the Temporal Memory component of the Hierarchical Temporal Memory (HTM) concept. Our implementation is built on a memristive crossbar array and is the result of a hardware-algorithm co-design process. Rather than using the memristive devices solely for data storage, our approach leverages their specific switching dynamics to propose a formulation of the peripheral circuitry, resulting in a more efficient design. By combining a brain-like algorithm with emerging non-volatile memristive device technology we strive for maximum energy efficiency. We present simulation results on the training of complex high-order sequences and discuss how the system is able to predict in a context-dependent manner.  Finally, we investigate the energy consumption during the training and conclude with a discussion of scaling prospects.

show abstract

Section: Discussionmentioning

confidence: 92%

System model of neuromorphic sequence learning on a memristive crossbar array

Siegel

Bouhadjar

Tetzlaff

et al. 2023

Neuromorph. Comput. Eng.

Self Cite

View full text Add to dashboard Cite

show abstract

“…A study by (Siegel et al 2023a) provided a specific instance of an electronic circuit design of the hardware implementing the different components of the spiking TM and showed in simulations that the system supports successful prediction performance. A follow-up study by (Siegel et al 2023b) taped out a memristive synaptic array of a simplified spiking TM model and showed successful performance on a simple sequence learning problem. A future study needs to upscale the array size and task difficulty.…”

Section: Discussionmentioning

confidence: 99%

Sequence learning in a spiking neuronal network with memristive synapses

Bouhadjar,

Siegel,

Tetzlaff

et al. 2023

Neuromorph. Comput. Eng.

Self Cite

View full text Add to dashboard Cite

Brain-inspired computing proposes a set of algorithmic principles that hold promise for advancing artificial intelligence. They endow systems with self learning capabilities, efficient energy usage, and high storage capacity. A core concept that lies at the heart of brain computation is sequence learning and prediction. This form of computation is essential for almost all our daily tasks such as movement generation, perception, and language. Understanding how the brain performs such a computation is not only important to advance neuroscience, but also to pave the way to new technological brain-inspired applications. A previously developed spiking neural network implementation of sequence prediction and recall learns complex, high-order sequences in an unsupervised manner by local, biologically inspired plasticity rules. An emerging type of hardware that may efficiently run this type of algorithm is neuromorphic hardware. It emulates the way the brain processes information and maps neurons and synapses directly into a physical substrate. Memristive devices have been identified as potential synaptic elements in neuromorphic hardware. In particular, redox-induced resistive random access memories (ReRAM) devices stand out at many aspects. They permit scalability, are energy efficient and fast, and can implement biological plasticity rules. In this work, we study the feasibility of using ReRAM devices as a replacement of the biological synapses in the sequence learning model. We implement and simulate the model including the ReRAM plasticity using the neural network simulator NEST. We investigate two types of ReRAM memristive devices: (i) a gradual, analog switching device, and (ii) an abrupt, binary switching device. We study the effect of different device properties on the performance characteristics of the sequence learning model, and demonstrate that, in contrast to many other artificial neural networks, this architecture is resilient with respect to changes in the on-off ratio and the conductance resolution, device variability, and device failure.

show abstract

“…[46] A detailed explanation of the chip layout has already been published. [47] A critical part for using resistive devices as logic memory is the mapping of a continuous resistance state R to a binary value 0/1. In most cases, a read-based state classification is exploited.…”

Section: Experimental Demonstration Of 1t-1r Logic Gatesmentioning

confidence: 99%

Improved Arithmetic Performance by Combining Stateful and Non‐Stateful Logic in Resistive Random Access Memory 1T–1R Crossbars

Brackmann,

Ziegler,

Jafari

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

Computing‐in‐memory (CIM) is a promising approach for overcoming the memory‐wall problem in conventional von‐Neumann architectures. This is done by performing certain computation tasks directly in the storage subsystem without transferring data between storage and processing units. Stateful and non‐stateful CIM concepts are recently attracting lots of interest, which are demonstrated as logical complete, energy efficient, and compatible with dense crossbar structures. However, sneak‐path currents in passive resistive random access memory (RRAM) crossbars degrade the operation reliability and require the usage of active 1 Transistor–1 Resistance (1T‐1R) bitcell designs. In this article, the arithmetic performance and reliability are investigated based on experimental measurements and variability‐aware circuit simulations. Herein, it is aimed for the evaluation of logic operations specifically with fully integrated 1T–1R crossbar devices. Based on these operations, an N‐bit full adder with optimized energy consumption and latency is demonstrated by combining stateful and non‐stateful CIM logic styles with regard to the specific conditions in active 1T–1R RRAM crossbars.

show abstract

Demonstration of neuromorphic sequence learning on a memristive array

Cited by 3 publications

References 11 publications

System model of neuromorphic sequence learning on a memristive crossbar array

System model of neuromorphic sequence learning on a memristive crossbar array

Sequence learning in a spiking neuronal network with memristive synapses

Improved Arithmetic Performance by Combining Stateful and Non‐Stateful Logic in Resistive Random Access Memory 1T–1R Crossbars

Contact Info

Product

Resources

About