Response : Computing with Neural Networks

Abstract

the flow of activity in a neural network (3) can be described by a simple potential function (4). Some computational problems can be transformed into a more-or-less equivalent optimization problem by a so-called regularization procedure (5,6). Hopfield's potential function provides the link between this optimization problem and its solution in terms of neural network, since in the network the potential is automatically minimized (1); and by proper choice of the network parameters, notably the neuronal interconnections, one can produce any potential function which is a polynomial of a second order (4). A new result is that, approximately, any minimization problem can be broken down into two minimization problems for second-order functionals (7), and thus can be solved by an appropriately constructed neural network.In particular for linear optimization problems this transformation into a second-order miuniization problem (8) and the corresponding network solution appear to be very natural (6,9). In general it is, however, not at all clear under which conditions this transformation of the original problem into a network computation is reasonably economic (for example, in terms of the size of the network). In many cases a quite different, more economic network solution may present itself immediately (10).From the point of view of economy it would be desirable to compare not only different network approaches with one another, but to compare network approaches to more conventional approaches to computation. Since these network approaches have a strong impact on analog computing and specialpurpose machines-as opposed to their effect

Keeping in mind the mind: Mental functions, networks and neurosurgery

“…Hopfield et al have modelled data processing particularly in the olfactory cortex [20,21]. In this area as well as the auditory cortex simple perceptrons are not as well suited to explain experimental facts.…”

“…McCulloch and Pitts, and later Hopfield stressed the almost unlimited computational power of NNMs [19,20,35]. The promise of these models has led to widespread application to almost any system of the brain [16,17,20,30,39,47].…”

“…The promise of these models has led to widespread application to almost any system of the brain [16,17,20,30,39,47]. In the following only some of the recent developments concerned with cognition and conscious awareness shall be discussed in some more detail.…”

The Possibility of Using Simple Neuron Models to Design Brain-Like Computers

“…In 1986, Hopfield and Tank proposed a very simple neuron model which takes the logistic function as its activation function [see Eq. (3)] (Hopfield& Tank, 1986 [3]). The Hopfield neuron model described by Eq.…”

Exponential input-to-state stability of recurrent neural networks with multiple time-varying delays

“…Due to its wide applications in various areas such as pattern classification, associative memory, parallel computation, optimization, moving object speed detection and so on, recurrent neural networks (RNNs) have been extensively studied by researchers in recent years (see, e.g., (Hopfield 1984;Hopfield and Tank 1986;Grujiá and Michel 1991;Matsouka 1992;Arik 2000;Ensari and Arik 2005;Zhang et al 2008;Wu et al 2008Wu et al , 2010Huang et al 2012;Huang and Feng 2009;Ahn 2010a;Liu and Cao 2010;Ahn 2010bAhn , c, 2011aAhn , b, 2012aSanchez and Perez 1999;Zhu and Shen 2012) and references therein). Since time-delay is unavoidably encountered in implementation of RNNs and is frequently a source of oscillation and instability, the stability of delayed neural networks has become a topic of great theoretical and practical importance, and many interesting results on stability in the Lyapunov sense have been derived (see also e.g.…”