An evolutionary approach to associative memory in recurrent neural networks

Abstract

Abstract. In this paper, we investigate the associative memory in recurrent neural networks, based on the model of evolving neural networks proposed by Nolfi, Miglino and Parisi.Experimentaily developed network has highly asymmetric synaptic weights and dilute connections, quite different from those of the Hopfield model.Some results on the effect of learning efficiency on the evolution are also presented. IntroductionAs a model of associative memory in terms of the recurrent-type artificial neural networks, Hopfield model [1] and its variants have been successfully developed and made some important results using the statistical mechanical techniques on the analogy with spin glass [2,3]. Their assumption of the symmetrically and fully connected weights of synapses is, however, unrealistic, since this does not arise naturally in biological neural systems. Some researches are found allowing asymmetric synaptic weights or diluting the network connections [2,3]. They do not concern the network structures, but most of them merely investigate the effects on the storage capacities under limited conditions.In any of above mentioned models, the physical positions where neurons exist or the distances between neurons are not considered at all. It must play an important role for the formation of biological networks in the brain. There should be the attempt to construct more natural models based on biologically and genetically plausible considerations, in which asymmetric and dilute connections are just the result. The genetic algorithm [4] can be a key ingredient toward such directions. Recently evolutionary search procedures have been combined with the artificial neural networks (so called "the evolutionary artificial neural networks") [5]. Almost all of the models deal with the feedforward multi-layered networks to raise the performance of the networks and in many of them the evolution of both weights and architectures is not integrated. In this paper we apply a genetic evolutionary scheme to recurrent neural networks and investigate associative memory. Aiming to construct a biologically inspired neural network, we take into account the fol!owing two points in network formation: 9 The correspondence of genotype and phenotype should not be direct. Only developmental rules of neurons are encoded in the genotype representation. 9 The information on the physical positions of neurons and the distances between neurons should be included in the model. The resulting network is a physical object in space. While the first point, the indirect encoding scheme, has been studied [6,7], few attempts have been made to incorporate both of the above two points, except for the model of "evolving (growing) neural networks r by Nolfi et al. [8,9]. So we follow their scheme treating a neural network as a creature in a physical environment.The outline of our model is as follows.(1) Prepare the individuals which have the genotypes encoded the information for generating networks. (2) Learn the patterns according to an appropriate learning rule. (3...

Differential Migration: Sensitivity analysis and comparison study

“…There are many papers reported using EC to replace the back-propagation learning algorithm in ANN in the past several years ( [1], [2], [7], [8], [9], [10], [12], [14], [18], [19], [20], [22], [26], [27], [28]). They showed that EC is a promising method for training ANN.…”