Mackey-Glass noisy chaotic time series prediction by a swarm-optimized neural network

López-Caraballo, C. H.; Salfate, Ignacio; Lazzús, Juan A.; Rojas, Pedro Díaz; Rivera, Marco; Palma-Chilla, L.

doi:10.1088/1742-6596/720/1/012002

Cited by 21 publications

(11 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, Zhang and Qi (2005) investigated the issue of how to effectively model artificial time series with deterministic behavior due to the existence of trend and seasonality using Artificial Neural Networks (ANNs). López-Caraballo et al (2016) examined ANNs on time series with noiseless and noisy chaotic behavior generated by Mackey-Glass series. Li and Lin (2016) applied the Self-Constructing Fuzzy Neural Network (SCFNN) on chaotic time series including Logistic and Henon data.…”

Section: Related Workmentioning

confidence: 99%

What is the best RNN-cell structure for forecasting each time series behavior?

Khaldi¹,

Afia²,

Chiheb³

et al. 2022

Preprint

View full text Add to dashboard Cite

It is unquestionable that time series forecasting is of paramount importance in many fields. The most used machine learning models to address time series forecasting tasks are Recurrent Neural Networks (RNNs).Typically, those models are built using one of the three most popular cells, ELMAN, Long-Short Term Memory (LSTM), or Gated Recurrent Unit (GRU) cells, each cell has a different structure and implies a different computational cost. However, it is not clear why and when to use each RNN-cell structure.Actually, there is no comprehensive characterization of all the possible time series behaviors and no guidance on what RNN cell structure is the most suitable for each behavior. The objective of this study is two-fold: it presents a comprehensive taxonomy of all-time series behaviors (deterministic, random-walk, nonlinear, long-memory, and chaotic), and provides insights into the best RNN cell structure for each time series behavior. We conducted two experiments: (1) The first experiment evaluates and analyzes the role of each component in the LSTM-Vanilla cell by creating 11 variants based on one alteration in its basic architecture (removing, adding, or substituting one cell component). ( 2) The second experiment evaluates and analyzes the performance of 20 possible RNN-cell structures. Our results showed that the MGU-SLIM3 cell is the most recommended for deterministic and nonlinear behaviors, the MGU-SLIM2 cell is the most suitable for random-walk behavior, FB1 cell is advocated for long-memory behavior, and LSTM-SLIM1 for chaotic behavior.

show abstract

Section: Related Workmentioning

confidence: 99%

What is the best RNN-cell structure for forecasting each time series behavior?

Khaldi¹,

Afia²,

Chiheb³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We consider the Mackey-Glass time series [10], which is used conventionally for benchmarking ESNs ([10, 17, 2, 13]. This one-dimensional series is described from the following delay differential equation (eq.…”

Section: Time Series Predictionmentioning

confidence: 99%

“…In our previous work it was shown that connectome-derived constraints could be applied to ESN reservoirs to improve model performance and variance (when compared to a conventional ESN) on the Mackey-Glass data set -a discrete, one-dimensional chaotic time series which is often used for benchmarking ESN performance [8][9][10][11][12][13]. In particular, when employing a full-stop replacement of the conventionally-used random ESN reservoir with a connectivity matrix derived from the fruit fly lateral horn ROI (region of interest), we observed a (roughly) 100-fold improvement in model variance in conjunction with improvements in performance (Mean-squared Error) on Mackey-Glass data sized at 800 and 2000 timesteps (with 200 samples left out of each for validation).…”

Section: Introductionmentioning

confidence: 99%

Using Connectome Features to Constrain Echo State Networks

Morra¹,

Daley²

2022

Preprint

View full text Add to dashboard Cite

We report an improvement to the conventional Echo State Network (ESN), which already achieves competitive performance in one-dimensional time series prediction of dynamical systems. Our model -a 20%-dense ESN with reservoir weights derived from a fruit fly connectome (and from its bootstrapped distribution) -yields superior performance on a chaotic time series prediction task, and furthermore alleviates the ESN's high-variance problem. We also find that an arbitrary positioning of weights can degrade ESN performance and variance; and that this can be remedied in particular by employing connectome-derived weight positions. Herein we consider four connectome features -namely, the sparsity, positioning, distribution, and clustering of weights -and construct corresponding model classes (A, B, B 2 , C) from an appropriate null model ESN; one with its reservoir layer replaced by a fruit fly connectivity matrix. After tuning relevant hyperparameters and selecting the best instance of each model class, we train and validate all models for multi-step prediction on size-variants (50, 250, 500, and 750 training input steps) of the Mackey-Glass chaotic time series; and compute their performance (Mean-Squared Error) and variance across train-validate trials.Preprint. Under review.

show abstract

“…Classification problems include Iris, diabetes diagnosis, thyroid disease, breast cancer, credit card, glass, heart, wine, page blocks, and liver disorders. Time series prediction problems include Mackey-Glass [10] and gas furnaces [11]. The number of features in the classification problem, the number of classes and the total number of samples listed in Table 1.…”

Section: Experiments 41 Defining Classification Problems and Predicting Time Seriesmentioning

confidence: 99%

Using the modified k-mean algorithm with an improved teaching-learning-based optimization algorithm for feedforward neural network training

Jouyban

Khorashadizade

2021

IJECE

View full text Add to dashboard Cite

In this paper we proposed a novel procedure for training a feedforward neural network. The accuracy of artificial neural network outputs after determining the proper structure for each problem depends on choosing the appropriate method for determining the best weights, which is the appropriate training algorithm. If the training algorithm starts from a good starting point, it is several steps closer to achieving global optimization. In this paper, we present an optimization strategy for selecting the initial population and determining the optimal weights with the aim of minimizing neural network error. Teaching-learning-based optimization (TLBO) is a less parametric algorithm rather than other evolutionary algorithms, so it is easier to implement. We have improved this algorithm to increase efficiency and balance between global and local search. The improved teaching-learning-based optimization (ITLBO) algorithm has added the concept of neighborhood to the basic algorithm, which improves the ability of global search. Using an initial population that includes the best cluster centers after clustering with the modified k-mean algorithm also helps the algorithm to achieve global optimum. The results are promising, close to optimal, and better than other approach which we compared our proposed algorithm with them.

show abstract

Mackey-Glass noisy chaotic time series prediction by a swarm-optimized neural network

Cited by 21 publications

References 24 publications

What is the best RNN-cell structure for forecasting each time series behavior?

What is the best RNN-cell structure for forecasting each time series behavior?

Using Connectome Features to Constrain Echo State Networks

Using the modified k-mean algorithm with an improved teaching-learning-based optimization algorithm for feedforward neural network training

Contact Info

Product

Resources

About