Continual Deep Learning for Time Series Modeling

Abstract

The multi-layer structures of Deep Learning facilitate the processing of higher-level abstractions from data, thus leading to improved generalization and widespread applications in diverse domains with various types of data. Each domain and data type presents its own set of challenges. Real-world time series data may have a non-stationary data distribution that may lead to Deep Learning models facing the problem of catastrophic forgetting, with the abrupt loss of previously learned knowledge. Continual learning is a paradigm of machine learning to handle situations when the stationarity of the datasets may no longer be true or required. This paper presents a systematic review of the recent Deep Learning applications of sensor time series, the need for advanced preprocessing techniques for some sensor environments, as well as the summaries of how to deploy Deep Learning in time series modeling while alleviating catastrophic forgetting with continual learning methods. The selected case studies cover a wide collection of various sensor time series applications and can illustrate how to deploy tailor-made Deep Learning, advanced preprocessing techniques, and continual learning algorithms from practical, real-world application aspects.

An Edge Computing Application of Fundamental Frequency Extraction for Ocean Currents and Waves

“…Probably of all the possible AI-based solutions, a neural network approach to calculation or classification is the most widely used in industry and research [ 24 ]. In the sensors research area, its use to process a time series of data is probably one of the most used.…”

“…However, despite the proven usefulness of frequency-based solutions, since most of the sensor provides time domain data, the first step to apply this type of approaches involves the use of the Fast Fourier Transform (FFT). By definition, the FFT is a computationally intensive operation, and that is the main reason to avoid its use [ 24 ].…”

“…Approaches in the literature that are implemented on microcontrollers rely on CNNs, RNNs, LSTM and their variations [ 24 ]. These applications are related, for example, to the estimation of Ion-Li battery parameters [ 5 ], structural health monitoring [ 28 ], rail vehicle running states [ 29 ], ECG Monitoring [ 30 ] or the eye blink detection [ 31 ].…”

Predictive Analysis of COVID-19 Epidemic in Thailand: Evaluating Control Lockdown Measures using LSTM Networks

“…Due to the complex and nonlinear nature of disease transmission under control measures, accurate adaptive models are essential. To address this gap, this study utilizes LSTM neural networks, renowned for their ability to capture long-term dependencies in infectious time-series data (Ao, & Fayek, 2023;Absar et al, 2022), and in other fields, for precise daily infection predictions. The details of the LSTM-based prediction model for daily infected cases are provided in Section 3.1.…”

The Time-Series Production Simulation in Cost Management of New Energy Grid Connection Under the Internet of Things

“…In the current context of the global energy transition, integrating new energy into the grid has become a key approach to addressing the balance of energy supply and demand and reducing carbon emissions [1]. However, grid connection management faces many challenges with the continuous expansion and diversification of new energy scales.…”