δ-NARMA neural networks: a new approach to signal prediction

Abstract

This article presents a new connectionist architecture for stochastic univariate signal prediction. After a review of related statistical and connectionist models pointing out their advantages and limitations, we introduce the-NARMA model as the simplest non-linear extension of ARMA models. These models then provide the units of a MLP-like neural network: the-NARMA neural network. The associated learning algorithm is based on an extension of classical back-propagation and on the concept of virtual error. Such networks can be seen as an extension of ARIMA and ARARMA models and faces the problem of non-stationary signal prediction. A theoretical study brings understanding of experimental phenomena observed during the-NARMA learning process. The experiments carried out on three railroad related real-life signals suggest that-NARMA networks outperform other studied univariate models.

Sequence Learning

“…The δ-NARMA neural network model (Bonnet et al, 1997c(Bonnet et al, , 1997b was designed for signal prediction. Let us consider a temporal information sequence, for instance the daily number of railroad travellers from Paris to Lyon.…”

Time in Connectionist Models

“…Jordan and Elman networks are typically used for memorizing (and recalling) sequences such as poems (sequences of words), robot arm movements (sequences of positions), etc... δ-NARMA. The δ-NARMA neural network model (Bonnet et al 1997c;Bonnet et al 1997b) was designed for signal prediction. Let us consider a temporal information sequence, for instance the daily number of railroad travellers from Paris to Lyon.…”