Entanglement-Structured LSTM Boosts Chaotic Time Series Forecasting

Abstract

Traditional machine-learning methods are inefficient in capturing chaos in nonlinear dynamical systems, especially when the time difference Δt between consecutive steps is so large that the extracted time series looks apparently random. Here, we introduce a new long-short-term-memory (LSTM)-based recurrent architecture by tensorizing the cell-state-to-state propagation therein, maintaining the long-term memory feature of LSTM, while simultaneously enhancing the learning of short-term nonlinear complexity. We stress that the global minima of training can be most efficiently reached by our tensor structure where all nonlinear terms, up to some polynomial order, are treated explicitly and weighted equally. The efficiency and generality of our architecture are systematically investigated and tested through theoretical analysis and experimental examinations. In our design, we have explicitly used two different many-body entanglement structures—matrix product states (MPS) and the multiscale entanglement renormalization ansatz (MERA)—as physics-inspired tensor decomposition techniques, from which we find that MERA generally performs better than MPS, hence conjecturing that the learnability of chaos is determined not only by the number of free parameters but also the tensor complexity—recognized as how entanglement entropy scales with varying matricization of the tensor.

Long-Term Prediction of Hydrometeorological Time Series Using a PSO-Based Combined Model Composed of EEMD and LSTM

“…LSTMs can effectively deal with the "long-term dependency" problem in RNNs, which can capture deeper connections in the sequences. Therefore, in recent years, LSTM has been widely used in time series forecasting [44,45].…”

“…In addition, the loss function used in all LSTM neural networks in this paper is the mean square error function. In order to ensure an optimal network performance, enhance convergence speed, and prevent gradient explosion, it is imperative to normalize the input data [45,49]. The normalization of the input data in this paper is carried out according to the following formula:…”

Percolation Theories for Quantum Networks

“…Rather, it suffices to simply store the vectors a, b, c, • • • of which the total size is DN. This, indeed, is how a tensor network works-by leveraging different ways of factorization that can be depicted through different graphical network structures [46]. Among various tensor networks, the matrix product state (MPS) is among the most researched [47].…”

Generalized relational tensors for chaotic time series

“…Such that in the article ( Yang, Krompass & Tresp, 2017 ) authors use Tensor-Train decomposition in order to deal with high-dimensional inputs, obtaining competitive results in comparison to classical approaches for dimensionality reduction. In Meng & Yang (2021) and Yu et al (2017) scholars apply tensorization in order to enhance RNN, LSTM models for long-term forecasting of chaotic time series. Using tensors for data representation proved to be effective in natural language problems.…”