Hardware implementation of Hierarchical Temporal Memory algorithm

Li, Weifu; Franzon, Paul D.

doi:10.1109/socc.2016.7905453

Cited by 6 publications

(15 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[20], 21 here we scaled only the number of mini-columns and the single pixel processing elements (total = 961x1) as detailed information about the distal segments and their sizes are not reported, and this results in 17.45 X improvement. In the case of fully CMOS digital design, 77.02 X is achieved when compared to our previous work in [17], and 31.75 X and 22.29 X when compared to the work done by Li Weifu et al [11], [18]. In contrast to other previous works, the power Fig.…”

Section: Power Consumption and Distributionmentioning

confidence: 64%

“…Fig. 11a illustrates the averaged MAPE over 5 runs for the HTM-HW prediction as a function of the faulty 18 device percentage for the aforementioned cases. It can be seen that the stuck-off fault has a positive marginal impact on the network performance as it leads to an increase in the network sparsity level.…”

Section: Device Failure and Network Robustnessmentioning

confidence: 99%

“…The authors also optimized their design further in [17]. Weifu Li et al [18], proposed a full architecture of the HTM algorithm in 2016. The proposed architecture is composed of 400 mini-columns (2 cells in each mini-column) connected in point-to-point format to the HTM input space, which eventually causes the mini-column to be in an active mode even when there is insignificant activity (noise) in the input space.…”

Section: Introductionmentioning

confidence: 99%

“…The authors in this paper also consider linear scaling for the network size and the power consumption. consumption reported in [18] does not consider the register files, which are usually the most power-hungry components in the design. In the case of [11], it is unclear if the register files power consumption is included.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Neuromorphic System for Spatial and Temporal Information Processing

Zyarah

Gómez

Kudithipudi

2020

IEEE Trans. Comput.

View full text Add to dashboard Cite

Neuromorphic systems that learn and predict from streaming inputs hold significant promise in pervasive edge computing and its applications. In this article, a neuromorphic system that processes spatio-temporal information on the edge is proposed. Algorithmically, the system is based on hierarchical temporal memory that inherently offers online learning, resiliency, and fault tolerance. Architecturally, it is a full custom mixed-signal design with an underlying digital communication scheme and analog computational modules. Therefore, the proposed system features reconfigurability, real-time processing, low power consumption, and low-latency processing. The proposed architecture is benchmarked to predict on real-world streaming data. The network's mean absolute percentage error on the mixed-signal system is 1.129 X lower compared to its baseline algorithm model. This reduction can be attributed to device non-idealities and probabilistic formation of synaptic connections. We demonstrate that the combined effect of Hebbian learning and network sparsity also plays a major role in extending the overall network lifespan. We also illustrate that the system offers 3.46 X reduction in latency and 77.02 X reduction in power consumption when compared to a custom CMOS digital design implemented at the same technology node. By employing specific low power techniques, such as clock gating, we observe 161.37 X reduction in power consumption.

show abstract

Section: Power Consumption and Distributionmentioning

confidence: 64%

Section: Device Failure and Network Robustnessmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Neuromorphic System for Spatial and Temporal Information Processing

Zyarah

Gómez

Kudithipudi

2020

IEEE Trans. Comput.

View full text Add to dashboard Cite

show abstract

“…Non-volatile HTM [12] Memrsitive HTM [13] LFSR HTM [25] Digital HTM [14] This work (PIM HTM) Spatial Spatial Spatial Spatial-temporal Spatial-Temporal a In the Digital HTM [14], the power of the register files is not included. b In these references, only the power of the SP is reported.…”

Section: Algorithmmentioning

confidence: 99%

Neuromorphic Architecture for the Hierarchical Temporal Memory

Zyarah

Kudithipudi

2019

IEEE Trans. Emerg. Top. Comput. Intell.

View full text Add to dashboard Cite

A biomimetic machine intelligence algorithm, that holds promise in creating invariant representations of spatiotemporal input streams is the hierarchical temporal memory (HTM). This unsupervised online algorithm has been demonstrated on several machine-learning tasks, including anomaly detection. Significant effort has been made in formalizing and applying the HTM algorithm to different classes of problems. There are few early explorations of the HTM hardware architecture, especially for the earlier version of the spatial pooler of HTM algorithm. In this article, we present a full-scale HTM architecture for both spatial pooler and temporal memory. Synthetic synapse design is proposed to address the potential and dynamic interconnections occurring during learning. The architecture is interweaved with parallel cells and columns that enable high processing speed for the HTM. The proposed architecture is verified for two different datasets: MNIST and the European number plate font (EUNF), with and without the presence of noise. The spatial pooler architecture is synthesized on Xilinx ZYNQ-7, with 91.16% classification accuracy for MNIST and 90% accuracy for EUNF, with noise. For the temporal memory sequence prediction, first and second order predictions are observed for a 5-number long sequence generated from EUNF dataset and 95% accuracy is obtained. Moreover, the proposed hardware architecture offers 1364X speedup over the software realization. These results indicate that the proposed architecture can serve as a digital core to build the HTM in hardware and eventually as a standalone self-learning system.

show abstract

An Overview of the Hierarchical Temporal Memory Accelerators

Zyarah¹,

Kudithipudi²

2023

Artificial Intelligence Applications and Reconfigurable Architectures

View full text Add to dashboard Cite

Hardware implementation of Hierarchical Temporal Memory algorithm

Cited by 6 publications

References 3 publications

Neuromorphic System for Spatial and Temporal Information Processing

Neuromorphic System for Spatial and Temporal Information Processing

Neuromorphic Architecture for the Hierarchical Temporal Memory

An Overview of the Hierarchical Temporal Memory Accelerators

Contact Info

Product

Resources

About