Neural modeling for time series: A statistical stepwise method for weight elimination

Abstract

Many authors use feedforward neural networks for modeling and forecasting time series. Most of these applications are mainly experimental, and it is often difficult to extract a general methodology from the published studies. In particular, the choice of architecture is a tricky problem. We try to combine the statistical techniques of linear and nonlinear time series with the connectionist approach. The asymptotical properties of the estimators lead us to propose a systematic methodology to determine which weights are nonsignificant and to eliminate them to simplify the architecture. This method (SSM or statistical stepwise method) is compared to other pruning techniques and is applied to some artificial series, to the famous Sunspots benchmark, and to daily electrical consumption data.

Artificial neural networks in bankruptcy prediction: General framework and cross-validation analysis

“…The most popular algorithm for training is the well-known backpropagation [54] which is basically a gradient steepest descent method with a constant step size. Due to problems of slow convergence and ineciency with the steepest descent method, many variations of backpropagation have been introduced for training neural networks [5,13,41]. Recently, Hung and Denton [27] and Subramanian and Hung [59] have proposed to use a general-purpose nonlinear optimizer, GRG2, in training neural networks.…”

Pre-processing inputs for optimally-configured time-delay neural networks

“…This is particularly important as some time series exhibit long-term cyclical behaviour, which is often ignored. Furthermore, as for other types of neural network, a TDNN cannot predict trend because the output of the network is typically computed using a threshold function, with the output of the function bounded to values, say, between 0 and 1 [7]. Autoregressive models, such as the seasonal autoregressive integrated moving average model (SARIMA), have been used to generate deseasonalised and de-trended time series, with the residuals used to train a TDNN, obtaining good results [2,3,4].…”

SONFIS: Structure Identification and Modeling with a Self-Organizing Neuro-Fuzzy Inference System

“…If the set V k is no longer empty, then the age k variable is set back to 0. After the application of the operators, all the unfrozen parameters (sub-networks) are updated according to equations (5), (8) and (9). Afterwards, the whole net architecture is frozen.…”

“…The users, based on their empirical intuitions and experience, are usually the ones that select the appropriate architecture to solve specific learning problems. For example, both the selection of the number of hidden neurons of a Feedforward Artificial Neural Network 7,9 and the activation functions in an Adaptive Network 13 are crucial and difficult decisions for system modeling in many real world problems. If the model size (complexity) is underestimated the model will not be able to effectively solve the problem, while an overly large size tends to overfit the training data and consequently results in poor generalization performance 28 .…”