Neuromemrisitive Architecture of HTM with On-Device Learning and Neurogenesis

Abstract

Hierarchical temporal memory (HTM) is a biomimetic sequence memory algorithm that holds promise for invariant representations of spatial and spatiotemporal inputs. This paper presents a comprehensive neuromemristive crossbar architecture for the spatial pooler (SP) and the sparse distributed representation classifier, which are fundamental to the algorithm. There are several unique features in the proposed architecture that tightly link with the HTM algorithm. A memristor that is suitable for emulating the HTM synapses is identified and a new Z-window function is proposed. The architecture exploits the concept of synthetic synapses to enable potential synapses in the HTM. The crossbar for the SP avoids dark spots caused by unutilized crossbar regions and supports rapid on-chip training within 2 clock cycles. This research also leverages plasticity mechanisms such as neurogenesis and homeostatic intrinsic plasticity to strengthen the robustness and performance of the SP. The proposed design is benchmarked for image recognition tasks using MNIST and Yale faces datasets, and is evaluated using different metrics including entropy, sparseness, and noise robustness. Detailed power analysis at different stages of the SP operations is performed to demonstrate the suitability for mobile platforms.

Neuromorphic System for Spatial and Temporal Information Processing

“…The encoder transforms sensory information into binary representations, while the classifiers map the HTM output to the corresponding class labels (SDR classifier) and identify anomalies (anomaly classifier). The mixed-signal design of the SDR classifier has been developed in our previous work [6]. Thus, this work will emphasize the design and implementation of a single HTM region, 2 which is equivalent to realizing the primary sensory region in the supragranular layers of the neocortex.…”

“…The output of the spatial pooler is a binary vector, which represents the joint activity of all mini-columns in the HTM region in response to the current input. The spatial pooler operation can be divided into three distinct phases: initialization, overlap and inhibition, and learning, discussed in our previous work [6] and briefly described below.…”

“…The mini-columns in HTM are responsible for capturing spatial patterns of the feedforward inputs. The HTM minicolumn circuit, developed in our previous work [6], is depicted in Fig. 3-(left).…”

“…Boosting is done via the use of a sense memristor (M s ). However, upon the completion of computing the overlap score (V aj a j ), its value, which is sampled by the sense memristor, is then presented to a WTA circuit (detailed description of the WTA is provided in [6]). The WTA performs a kmax operation on V aj ; 8j followed by a thresholding, to generate the final jth mini-column output, (L j ), as given in (1) and (2):…”

“…Biologically inspired systems, such as hierarchical temporal memory (HTM) [1], [2], have demonstrated strong capability in processing spatial and temporal information with a high degree of plasticity while learning models of the world. HTM also exhibits natural compatibility for continuous online learning [3], noise and fault tolerance [4], and low power consumption achieved through sparse neuronal activity [5], [6]. These properties make the algorithm attractive for a wide range of applications such as visual object recognition and classification [7], [8], prediction of data streams [9], natural language processing and anomaly detection [10].…”