Hopfield Networks, Simulated Annealing, and Chaotic Neural Networks

Abstract

The Hopfield model [27,28] is the most popular dynamic model. It is biologically plausible since it functions like the human retina [36]. It is a fully interconnected recurrent network with J McCulloch-Pitts neurons. The Hopfield model is usually represented by using a J -J layered architecture, as illustrated in Fig. 6.1. The input layer only collects and distributes feedback signals from the output layer. The network has a symmetric architecture with a symmetric zero-diagonal real weight matrix, that is, w i j = w ji and w ii = 0. Each neuron in the second layer sums the weighted inputs from all the other neurons to calculate its current net activation net i , then applies an activation function to net i and broadcasts the result along the connections to all the other neurons. In the figure, w ii = 0 is represented by a dashed line; φ(·) and θ are, respectively, a vector comprising the activation functions for all the neurons and a vector comprising the biases for all the neurons.The Hopfield model operates in an unsupervised manner. The dynamics of the network are described by a system of nonlinear ordinary differential equations. The discrete form of the dynamics is defined by1)where net i is the weighted net input of the ith neuron, x i (t) is the output of the ith neuron, θ i is a bias to the neuron, and φ(·) is the sigmoidal function.

Artificial Intelligence: A Survey on Evolution and Future Trends

“…Neural networks have a great experience to accomplish many complicated things, for example, language recognition; there is an article published in 2019 about the use of artificial neural networks in Turkish character (Türkçe karakterler) recognition written with a mouse [15]. There are also convolutional neural networks [16], Kohonen Self Organizing [17], Boltzmann machine networks [18], and Hopfield networks [19]. Picking the right network for a particular task depends on the type of data to be trained and the job done with it.…”