Recurrent Correlation Associative Memories: A Feature Space Perspective

Abstract

In this paper, we analyze a model of recurrent kernel associative memory (RKAM) recently proposed by Garcia and Moreno. We show that this model consists in a kernelization of the recurrent correlation associative memory (RCAM) of Chiueh and Goodman. In particular, using an exponential kernel, we obtain a generalization of the well-known exponential correlation associative memory (ECAM), while using a polynomial kernel, we obtain a generalization of higher order Hopfield networks with Hebbian weights. We show that the RKAM can outperform the aforementioned associative memory models, becoming equivalent to them when a dominance condition is fulfilled by the kernel matrix. To ascertain the dominance condition, we propose a statistical measure which can be easily computed from the probability distribution of the interpattern Hamming distance or directly estimated from the memory vectors. The RKAM can be used below saturation to realize associative memories with reduced dynamic range with respect to the ECAM and with reduced number of synaptic coefficients with respect to higher order Hopfield networks.

An Introduction to Quaternion-Valued Recurrent Projection Neural Networks

“…In contrast to the original Hopfield neural network which has a limited storage capacity, some RCNN models can reach the storage capacity of an ideal associative memory [43]. Furthermore, certain RCNNs can be viewed as kernelized versions of the Hopfield network with Hebbian learning [16,17,18]. Finally, RCNNs are closely related to the dense associative memory model introduced by Krotov and Hopfield to establish the duality between associative memories and deep learning [44,45].…”

A new class of multi-stable neural networks: Stability analysis and learning process

“…Dynamical Neural Networks (NN) are powerful methods which are applied in system identification and modeling nonlinear dynamical systems (Nørgaard, 2000;Janczak, 2005;Liu, 2001), classification of time series (Ao, 2010;Hu & Hwang, 2010) and pattern recognition using memories (Zurada, 1992;Perfetti & Ricci, 2008;Wang et al, 1990;Shen & Cruz, 2005;Chartier & Boukadoum, 2006;Sudo et al, 2009). Within neurobiological aspect, these methods are important to emulate and explain different biological behaviors includes information storage and recall (Zeng and Zheng, 2012).…”

Fully Binary Neural Network Model and Optimized Hardware Architectures for Associative Memories

“…Moreover, these associative memories are robust against input noise and have a practically constant retrieval time independent of the number of stored associations. These memories have been intensively studied in the past (e.g., Kohonen [1977], Palm [1980Palm [ , 2013, and Willshaw [1971]) with many successful applications (e.g., Sudo et al [2009], Perfetti and Ricci [2008], and Annovi et al [2013]). …”