A study of pattern recovery in recurrent correlation associative memories

Abstract

Abstract-In this paper, we analyze the recurrent correlation associative memory (RCAM) model of Chiueh and Goodman. This is an associative memory in which stored binary memory patterns are recalled via an iterative update rule. The update of the individual pattern-bits is controlled by an excitation function, which takes as its arguement the inner product between the stored memory patterns and the input patterns. Our contribution is to analyze the dynamics of pattern recall when the input patterns are corrupted by noise of a relatively unrestricted class. We make three contributions. First, we show how to identify the excitation function which maximizes the separation (the Fisher discriminant) between the uncorrupted realization of the noisy input pattern and the remaining patterns residing in the memory. Moreover, we show that the excitation function which gives maximum separation is exponential when the input bit-errors follow a binomial distribution. Our second contribution is to develop an expression for the expectation value of bit-error probability on the input pattern after one iteration. We show how to identify the excitation function which minimizes the bit-error probability. However, there is no closed-form solution and the excitation function must be recovered numerically. The relationship between the excitation functions which result from the two different approaches is examined for a binomial distribution of bit-errors. The final contribution is to develop a semiempirical approach to the modeling of the dynamics of the RCAM. This provides us with a numerical means of predicting the recall error rate of the memory. It also alllows us to develop an expression for the storage capacity for a given recall error rate.Index Terms-Associative memory, error rates, recurrent correlation associative memory (RCAM), storage capacity.

A Weighted Voting Model of Associative Memory

“…For notational convenience, let rank . Hence, the total number of votes received by pattern is given by (38) The probabilities , , can be computed from training data by running trial simulations. If no training data is available, the following heuristic setting is found to be Fig.…”

“…Note that (38) shows how the local probabilities are combined to produce a global measure that is the target pattern. Again, the combination scheme is the simple summing given in (34) (the only difference is that the scaling factor has been dropped here).…”

“…Memory pattern #1 has , , and . Using (38), the weighted vote assigned to memory pattern #1 is given by which is the same as adding all the local probabilities for memory pattern #1 given in Fig. 4.…”

“…In fact, this focus on just the first two requirements is characteristic of more recent research as well [10], [30], [31], [38]. For many practical applications, though, the third requirement is as important or even more important than the first two.…”

Recurrent Correlation Associative Memories: A Feature Space Perspective

“…In this network, the convergence time is almost unaffected by the number of stored vectors, hence, while space complexity is still , the recall time is independent of . When the nonlinear function is an exponential, we obtain the exponential correlation associative memory (ECAM), which has been deeply investigated both analytically and through simulations by several authors [13]- [18]. It has been shown that the ECAM can reach the capacity of an ideal AM, which is exponential with respect to the length of memory vectors [16].…”