Long-term properties of time series generated by a perceptron with various transfer functions

Abstract

We study the effect of various transfer functions on the properties of a time series generated by a continuous-valued feed-forward network in which the next input vector is determined from past output values. The parameter space for monotonic and non-monotonic transfer functions is analyzed in the unstable regions with the following main finding; non-monotonic functions can produce robust chaos whereas monotonic functions generate fragile chaos only. In the case of nonmonotonic functions, the number of positive Lyapunov exponents increases as a function of one of the free parameters in the model, hence, high dimensional chaotic attractors can be generated. We extend the analysis to a combination of monotonic and non-monotonic functions.

Generation of unpredictable time series by a neural network

“…This attractor is robust to noise [19]. The time series displays chaotic behaviour only for very special choices of w and β ("fragile chaos") if the transfer function is monotonic, and for generic initial conditions ("robust chaos") only if it is non-monotonic [20]. We will compare these properties to those of the CSG.…”

Time series prediction by feedforward neural networks is it difficult?

“…A quasiperiodic solution means that the solution at each step changes and one cannot find one point in the return map-the space that is defined by S t i versus S t i−1 -that satisfies the update equations. However, there is a certain line in the attractor dimension that the solutions are confined to (see [4][5][6] for more details). Such a line indicates problems in learning.…”

“…In modern research, different methods taken from a variety of fields are employed for this task [1]. Time series that are produced by neural networks have lately been studied in the framework of the statistical physics field [2][3][4][5][6]. One of the novel findings regarding time series produced by neural networks is concerned with series produced by perceptrons with continuous activation function.…”

“…a student trained on QPSs obtains very little information about the teacher. However, a host of simulation results covering most of the parameter space show that the student who learned only partially can predict the same sequence over many steps ahead [5,6].…”

Chaos in Diluted Networks With Continuous Neurons

“…Considering neural networks in the more general frame of input-output devices, neural nets with nonmonotonic transfer functions have been employed in applications such as generation of robust chaos [17,18] and clustering of data [19].…”