Untitled

Bertels, Koen; Neuberg, Luc; Vassiliadis, S.; Pechanek, D. G.

doi:10.1023/a:1009680311307

Cited by 6 publications

(2 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As we can observe from this figure (going from upper left to lower right), not only fixed points, limit cycles, bifurcations, and Bifurcation diagram for T = 0.065, 0.35, and 1.25 (Reprinted with permission from Ref. [4]).…”

Section: The Role Of Betamentioning

confidence: 99%

See 1 more Smart Citation

A look inside the learning process of neural networks

et al. 2000

Self Cite

View full text Add to dashboard Cite

ne of the best-known results of the sciences of complexity is that complex systems learn on the edge of chaos, by which is meant that both chaotic and orderly states coexist and that the system remains close to this borderline and may switch from one state to the other. In this article, we take a look inside the learning process of neural networks, and we more specifically focus on the role of chaos for learning a nonlinearly separable function, the XOR Boolean function. It is clear that neural networks are complex adaptive systems (CAS) in the sense of Casti's definition of CAS [1]. The neurones are individual components that can make some kind of computation. These neurones interact as they send the result of their computation to neighboring neurones, and they only use local information as they do not receive information from all the neurones in the network. For a detailed discussion of neural networks, we refer to Ref. [2], which previously appeared in Complexity, and for an extensive discussion of the results reported here we refer to Refs.[3], [4], and [5]. The following restrictions apply to the research reported on in this article. We only look at a relatively simple problem. Even though it could be argued that a simple problem such as the XOR problem is not representative of more realistic problems, the simplicity of the network allows us to better focus on the phenomenon we want to study, namely the occurrence of chaos during the learning process. We also restrict ourselves to multilayer neural networks that are trained, using the backpropagation algorithm. Similar analyses for other learning algorithms could be the subject of further research.The article is structured as follows. We first briefly introduce the basic equations of the backpropagation algorithm (BPA). We then show that chaos occurs during the learning process and investigate in more detail the role of two parameters, the learning rate and the temperature. We finally address the question whether chaos increases learning speed and could therefore be considered necessary or beneficial for efficient learning. ALGORITHMIC BASICSWe will not explain the entire algorithm here as it has already been discussed previously, but the remainder of the article centers around the so-called Delta rule. As we know, backpropagation neural networks belong to the category of supervised learning neural networks, implying that it is known what output the network has to produce for a given set of inputs. The following equations allow us to compute the output of any node (hidden or output

show abstract

Section: The Role Of Betamentioning

confidence: 99%

“…In Ref. [4] it was shown not only that chaos occurs for different values of ␤ but also that the critical point, called ␤ c , at which chaos occurs differs in function of the value of the temperature parameter: The higher the temperature, the higher the value of ␤ c . We will now first explain why chaos occurs.…”

Section: Figurementioning

confidence: 99%