Neuromorphic Architecture for the Hierarchical Temporal Memory

Abstract

A biomimetic machine intelligence algorithm, that holds promise in creating invariant representations of spatiotemporal input streams is the hierarchical temporal memory (HTM). This unsupervised online algorithm has been demonstrated on several machine-learning tasks, including anomaly detection. Significant effort has been made in formalizing and applying the HTM algorithm to different classes of problems. There are few early explorations of the HTM hardware architecture, especially for the earlier version of the spatial pooler of HTM algorithm. In this article, we present a full-scale HTM architecture for both spatial pooler and temporal memory. Synthetic synapse design is proposed to address the potential and dynamic interconnections occurring during learning. The architecture is interweaved with parallel cells and columns that enable high processing speed for the HTM. The proposed architecture is verified for two different datasets: MNIST and the European number plate font (EUNF), with and without the presence of noise. The spatial pooler architecture is synthesized on Xilinx ZYNQ-7, with 91.16% classification accuracy for MNIST and 90% accuracy for EUNF, with noise. For the temporal memory sequence prediction, first and second order predictions are observed for a 5-number long sequence generated from EUNF dataset and 95% accuracy is obtained. Moreover, the proposed hardware architecture offers 1364X speedup over the software realization. These results indicate that the proposed architecture can serve as a digital core to build the HTM in hardware and eventually as a standalone self-learning system.

Neuromorphic System for Spatial and Temporal Information Processing

“…[20], 21 here we scaled only the number of mini-columns and the single pixel processing elements (total = 961x1) as detailed information about the distal segments and their sizes are not reported, and this results in 17.45 X improvement. In the case of fully CMOS digital design, 77.02 X is achieved when compared to our previous work in [17], and 31.75 X and 22.29 X when compared to the work done by Li Weifu et al [11], [18]. In contrast to other previous works, the power Fig.…”

“…4. The feasibility of the HTM network scaling (beyond 1024 minicolumns) can be made possible by adopting the slicing approach proposed in our previous work [17].…”

“…Comparatively, this is %24 times more than a conventional network with no sparse activities, and X 16 times more than the SNNs. 17 In spite of the fact that SNNs are asynchronous and sparse in nature, usually their neurons fire and their synaptic connections are tuned multiple times while processing a single input. This is because each input is stochastically encoded as a stream of spikes.…”

“…Furthermore, it supports synthetic synapses, which are realized with distributed memory blocks, to enable synaptic pathway dynamics. The authors also optimized their design further in [17]. Weifu Li et al [18], proposed a full architecture of the HTM algorithm in 2016.…”

A Temporal Pool Learning Algorithm Based on Location Awareness

“…In this model, the backtracking algorithm proposed by the author makes the prediction of the temporal pool more stable. In recent years, some scholars start to implement TPL on hardware [27]. Many scholars combine SPL with TPL and apply HTM to many fields with time series data, such as anomaly detection of time series data [28], heart attack detection [29], medical data flow prediction [25], hydrological intelligent monitoring [30], abnormal ECG detection [31], and abnormal detection in crowd management [32].…”