On Temporal Generalization of Simple Recurrent Networks

Convergence of gradient method for Elman networks

“…This feedback mechanism allows Elman networks to learn to recognize and generate not only spatial patterns but also temporal patterns [3] . The Elman network has been proven powerful for classifying time series data [4] and for modeling linear and nonlinear dynamical systems [5] .…”

Nonlinear stable adaptive control based upon Elman networks

“…However,the training of FRNN weight values is much more complicated[2] than that of MFNN,therefore this restriction can't bring FRNN's whole advantage into play in modeling system dynamics and real-time control. In recent years,finding a kind of simple recurrent networks model to fit the nonlinear control has become a new research hotspot[3'4].Elman networks[5] unique charmhas attacted lots of researchers' attentions, and wide slope researches have been developed in aspects such as nonlinear modeling,temporal signal process and automata theory [6][7][8]. In [6] the dynamic backpropagation algorithm (DBP) was applied to trainElman networks' weight values for modeling nonlinear dynamic systems.…”

“…Elman networks[5] unique charmhas attacted lots of researchers' attentions, and wide slope researches have been developed in aspects such as nonlinear modeling,temporal signal process and automata theory [6][7][8]. In [6] the dynamic backpropagation algorithm (DBP) was applied to trainElman networks' weight values for modeling nonlinear dynamic systems.…”

Convergence of Gradient Descent Algorithm with Penalty Term For Recurrent Neural Networks

“…These feedback mechanisms allow RNN to learn to recognize and generate not only temporal patters, but also spatial patterns [3]. As in the case of feedforward neural networks, the gradient methods are proposed for training RNN [2] due to its simplicity, either in an off-line(batch) or an online(incremental) manner.…”