A hierarchical bayesian model of invariant pattern recognition in the visual cortex

George, Dileep; Hawkins, Jeff

doi:10.1109/ijcnn.2005.1556155

Cited by 147 publications

(175 citation statements)

References 16 publications

Supporting

Mentioning

170

Contrasting

Unclassified

Order By: Relevance

“…[5]) we chose to test our modified spatial poolers, ASP+M and ASP+G, and ICA on the well known MNIST handwritten digits dataset [12]. The set comprises 70,000 handwritten images (28 × 28 pixels) of the digits 0-9, split into 60,000 training images and 10,000 test images.…”

Section: Experimental Evaluation and Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluating Sparse Codes on Handwritten Digits

Main

Cowley

Kneller

et al. 2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Sparse coding of visual information has been of interest to the neuroscientific community for many decades and it is widely recognised that sparse codes should exhibit a high degree of statistical independence, typically measured by the kurtosis of the response distributions. In this paper we extend work on the hierarchical temporal memory model by studying the suitability of the augmented spatial pooling (ASP) sparse coding algorithm in comparison with independent component analysis (ICA) when applied to the recognition of handwritten digits. We present an extension to the ASP algorithm that forms synaptic receptive fields located closer to their respective columns and show that this produces lower Naïve Bayes classification errors than both ICA and the original ASP algorithm. In evaluating kurtosis as a predictor of classification performance, we also show that additional measures of dispersion and mutual information are needed to reliably distinguish between competing approaches.

show abstract

Section: Experimental Evaluation and Discussionmentioning

confidence: 99%

“…In the original ASP algorithm [19] the synapse permanence values have a potential range of 0 to 1 with the threshold set at 0.2 and initial permanences bound to be within 0.1 of the threshold. We use these values in our current study (see lines [4][5].…”

Section: Algorithm 1 Initialisecolumns(columns Method M)mentioning

confidence: 99%

Evaluating Sparse Codes on Handwritten Digits

Main

Cowley

Kneller

et al. 2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Following the publication of Hawkins' memory-prediction theory of brain function [8], supported by Mountcastle's unit model of computational processes across the neocortex [11], George and Hawkins implemented an initial mathematical model of the Hierarchical Temporal Memory (HTM) concept and applied it to a simple pattern recognition problem as a successful proof of concept [4]. This partial HTM implementation was based on a hierarchical Bayesian network, modeling invariant pattern recognition behavior in the visual cortex.…”

Section: Related Workmentioning

confidence: 99%

Evolving Hierarchical Temporal Memory-Based Trading Models

Gabrielsson

König

Johansson

2013

Applications of Evolutionary Computation

View full text Add to dashboard Cite

Abstract. We explore the possibility of using the genetic algorithm to optimize trading models based on the Hierarchical Temporal Memory (HTM) machine learning technology. Technical indicators, derived from intraday tick data for the E-mini S&P 500 futures market (ES), were used as feature vectors to the HTM models. All models were configured as binary classifiers, using a simple buy-and-hold trading strategy, and followed a supervised training scheme. The data set was partitioned into multiple folds to enable a modified cross validation scheme. Artificial Neural Networks (ANNs) were used to benchmark HTM performance. The results show that the genetic algorithm succeeded in finding predictive models with good performance and generalization ability. The HTM models outperformed the neural network models on the chosen data set and both technologies yielded profitable results with above average accuracy. IntroductionOne recent machine learning technology that has shown good potential in algorithmic trading is Hierarchical Temporal Memory (HTM). It borrows concepts from neural networks, Bayesian networks and makes use of spatiotemporal clustering algorithms to handle noisy inputs and to create invariant representations of patterns discovered in its input stream. In a previous paper [3], an initial study was carried-out where the predictive performance of the HTM technology was investigated within algorithmic trading of financial markets. The study showed promising results, in which the HTMbased algorithm was profitable across bullish-, bearish and horizontal market trends, yielding comparable results to its neural network benchmark. Although, the previous work lacked any attempt to produce near optimal trading models. One popular group of optimization methods are evolutionary optimization methods. The simplest evolutionary optimization technique is the genetic algorithm. This paper extends the HTM-based trading algorithm, developed in the previous work, by employing the genetic algorithm as an optimization method. Once again, neural networks are used as the benchmark technology.

show abstract

“…George and Hawkins [8] have also used a hierarchical Bayesian network to perform pattern recognition on line drawings. This work was a partial realization of a more ambitious project based on Hawkins' model of the neocortex proposed in [9] (now known as hierarchical temporal memory or HTM).…”

Section: Introductionmentioning

confidence: 99%

“…In fact, Hawkins sees the fundamental task of neocortical processing as prediction and so places temporal change at the centre of his model. This emphasis on time was only partly realized in George and Hawkins original work [8] where movies of the line drawings were displayed and the system was "primed" to expect the repetition of the previous image. More recently, a new implementation of HTM has been developed that explicitly includes a temporal pooling component within a vector quantization framework [11].…”

Section: Introductionmentioning

confidence: 99%

Character Recognition Using Hierarchical Vector Quantization and Temporal Pooling

Thornton

Faichney

Blumenstein

et al. 2008

AI 2008: Advances in Artificial Intelligence

View full text Add to dashboard Cite

Abstract. In recent years, there has been a cross-fertilization of ideas between computational neuroscience models of the operation of the neocortex and artificial intelligence models of machine learning. Much of this work has focussed on the mammalian visual cortex, treating it as a hierarchically-structured pattern recognition machine that exploits statistical regularities in retinal input. It has further been proposed that the neocortex represents sensory information probabilistically, using some form of Bayesian inference to disambiguate noisy data. In the current paper, we focus on a particular model of the neocortex developed by Hawkins, known as hierarchical temporal memory (HTM). Our aim is to evaluate an important and recently implemented aspect of this model, namely its ability to represent temporal sequences of input within a hierarchically structured vector quantization algorithm. We test this temporal pooling feature of HTM on a benchmark of cursive handwriting recognition problems and compare it to a current state-of-the-art support vector machine implementation. We also examine whether two pre-processing techniques can enhance the temporal pooling algorithm's performance. Our results show that a relatively simple temporal pooling approach can produce recognition rates that approach the current state-of-the-art without the need for extensive tuning of parameters. We also show that temporal pooling performance is surprisingly unaffected by the use of preprocessing techniques.

show abstract

A hierarchical bayesian model of invariant pattern recognition in the visual cortex

Cited by 147 publications

References 16 publications

Evaluating Sparse Codes on Handwritten Digits

Evaluating Sparse Codes on Handwritten Digits

Evolving Hierarchical Temporal Memory-Based Trading Models

Character Recognition Using Hierarchical Vector Quantization and Temporal Pooling

Contact Info

Product

Resources

About