Traditional neural networks require enormous amounts of data to build their complex mappings during a slow training procedure that hinders their abilities for relearning and adapting to new data. Memory-augmented neural networks enhance neural networks with an explicit memory to overcome these issues. Access to this explicit memory, however, occurs via soft read and write operations involving every individual memory entry, resulting in a bottleneck when implemented using the conventional von Neumann computer architecture. To overcome this bottleneck, we propose a robust architecture that employs a computational memory unit as the explicit memory performing analog in-memory computation on high-dimensional (HD) vectors, while closely matching 32-bit software-equivalent accuracy. This is achieved by a content-based attention mechanism that represents unrelated items in the computational memory with uncorrelated HD vectors, whose real-valued components can be readily approximated by binary, or bipolar components. Experimental results demonstrate the efficacy of our approach on few-shot image classification tasks on the Omniglot dataset using more than 256,000 phase-change memory devices. Our approach effectively merges the richness of deep neural network representations with HD computing that paves the way for robust vector-symbolic manipulations applicable in reasoning, fusion, and compression.

Brain-inspired hyperdimensional (HD) computing models neural activity pa erns of the very size of the brain's circuits with points of a hyperdimensional space, that is, with hypervectors. Hypervectors are D-dimensional (pseudo)random vectors with independent and identically distributed (i.i.d.) components constituting ultra-wide holographic words: D = 10, 000 bits, for instance. At its very core, HD computing manipulates a set of seed hypervectors to build composite hypervectors representing objects of interest. It demands memory optimizations with simple operations for an e cient hardware realization. In this paper, we propose hardware techniques for optimizations of HD computing, in a synthesizable open-source VHDL library, to enable co-located implementation of both learning and classi cation tasks on only a small portion of Xilinx ® UltraScale ™ FPGAs: (1) We propose simple logical operations to rematerialize the hypervectors on the y rather than loading them from memory. ese operations massively reduce the memory footprint by directly computing the composite hypervectors whose individual seed hypervectors do not need to be stored in memory.(2) Bundling a series of hypervectors over time requires a multibit counter per every hypervector component. We instead propose a binarized back-to-back bundling without requiring any counters. is truly enables on-chip learning with minimal resources as every hypervector component remains binary over the course of training to avoid otherwise multibit components. (3) For every classi cation event, an associative memory is in charge of nding the closest match between a set of learned hypervectors and a query hypervector by using a distance metric. is operator is proportional to hypervector dimension (D), and hence may take O(D) cycles per classi cation event. Accordingly, we signi cantly improve the throughput of classi cation by proposing associative memories that steadily reduce the latency of classi cation to the extreme of a single cycle. (4) We perform a design space exploration incorporating the proposed techniques on FPGAs for a wearable biosignal processing application as a case study. Our techniques achieve up to 2.39× area saving, or 2337× throughput improvement. e Pareto optimal HD architecture is mapped on only 18340 con gurable logic blocks (CLBs) to learn and classify ve hand gestures using four electromyography sensors.

The typically nonlinear and asymmetric response of synaptic memristors to positive and negative electrical pulses makes the realization of accurate deep neural networks very challenging. Here, we integrate a two-terminal valence change memory (VCM) into a photonic/plasmonic circuit and show that the switching properties of this memristor become more gradual and symmetric under light irradiation. The added optical input acts on the VCM as a third, independent modulation channel. It locally heats the active area of the device, which enhances the generation of oxygen vacancies and broadens the resulting nanoscale conductive filaments. The measured conductance modulation of the VCM is then inserted into a neural network simulator. Using the MNIST data set of handwritten digits as an application, a light-enhanced recognition accuracy of 93.53% is demonstrated, similar to ideally performing memristors (94.86%) and much higher than those without light (67.37%). Notably, the optical signal does not increase the overall energy consumption by more than 3.2%. Finally, an approach to scale up our electro-optical technology is proposed, which could allow high-density, energy-efficient neuromorphic computing chips.

No abstract