Time-series anomaly detection with stacked Transformer representations and 1D convolutional network

Kim, Jina; Kang, Hyeongwon; Kang, Pilsung

doi:10.1016/j.engappai.2023.105964

Cited by 45 publications

(17 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Transformer-based methods [15], [38] that capture temporal dependencies in sequential data have been proposed in various fields. Jina et al [38] proposed a time-series anomaly detection method that employs a stacked transformer encoder and one-dimensional convolutional neural network (1D-CNN) [39]-based decoder, to capture global trends and local variability in time-series data. ALSP [15] is a stock selection method that uses a transformer encoder to capture long-and short-term stock patterns.…”

Section: Transformer-based Methodsmentioning

confidence: 99%

Diversified Adaptive Stock Selection Using Continual Graph Learning and Ensemble Approach

Kim,

Lee

2024

IEEE Access

View full text Add to dashboard Cite

Stock selection is essential for portfolio diversification to reduce risks and maximize profits. However, stock selection is difficult owing to the non-stationary nature of stock markets. In fact, stock markets experience abrupt or gradual concept drift because of their inherent volatility. Concept drift is a phenomenon in which the statistical characteristics of data change over time. Recent stock selection methods have adopted graph neural networks to capture the relational dependencies between stocks. These methods perform non-continual learning that uses a fixed set of stocks without knowledge retention. Noncontinual graph learning-based methods can adapt to abrupt concept drift, while continual graph learningbased methods can adapt to gradual concept drift because they involve knowledge retention. To adapt to both abrupt and gradual concept drifts, we propose a stock selection framework called DASS, which combines non-continual and continual models for diversified adaptation. For both models, we employ graph learning to extract both temporal and relational dependencies. Our graph learning method relies on three main components: (1) low-level temporal modeling, which extracts temporal dependencies of individual stocks, (2) relational modeling, which extracts relational dependencies between stocks, and (3) high-level temporal modeling, which extracts temporal dependencies from the learned relational dependencies. Furthermore, DASS constructs simple graphs and hypergraphs based on dynamic time warping of stock prices and volume data. The performance of DASS is compared with that of state-of-the-art stock selection methods. Experimental results for stocks included in the Standard & Poor's 500 index reveal that DASS achieves a compounded annual growth rate of 83.2%, outperforming the second-best method by 23.0%P.

show abstract

Section: Transformer-based Methodsmentioning

confidence: 99%

Diversified Adaptive Stock Selection Using Continual Graph Learning and Ensemble Approach

Kim,

Lee

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…To make full use of temporal patterns, a frequency attention module is designed to extract periodic oscillation features. Kim, J. et al [28] proposed an unsupervised prediction-based time series anomaly detection method using the transformer, which learns the dynamic patterns of sequential data through a self-attentive mechanism. The output representation of each transformer layer is accumulated in the encoder to obtain a representation with multiple levels and rich information.…”

Section: The Methods For Spatial-temporal Correlation Fusionmentioning

confidence: 99%

DCFF-MTAD: A Multivariate Time-Series Anomaly Detection Model Based on Dual-Channel Feature Fusion

Yang

Gao

et al. 2023

Sensors

View full text Add to dashboard Cite

The detection of anomalies in multivariate time-series data is becoming increasingly important in the automated and continuous monitoring of complex systems and devices due to the rapid increase in data volume and dimension. To address this challenge, we present a multivariate time-series anomaly detection model based on a dual-channel feature extraction module. The module focuses on the spatial and time features of the multivariate data using spatial short-time Fourier transform (STFT) and a graph attention network, respectively. The two features are then fused to significantly improve the model’s anomaly detection performance. In addition, the model incorporates the Huber loss function to enhance its robustness. A comparative study of the proposed model with existing state-of-the-art ones was presented to prove the effectiveness of the proposed model on three public datasets. Furthermore, by using in shield tunneling applications, we verify the effectiveness and practicality of the model.

show abstract

“…This is because each subsequence undergoes the same input modification. Temporal translation invariance [29] allows for a pattern learned at one point in a sequence to be recognized at a different location later.…”

Section: Pv Power Prediction Model 221 One-dimensional Convolutionmentioning

confidence: 99%

A Multi-Step-Ahead Photovoltaic Power Forecasting Approach Using One-Dimensional Convolutional Neural Networks and Transformer

Moon

2024

Electronics

View full text Add to dashboard Cite

Due to environmental concerns about the use of fossil fuels, renewable energy, especially solar energy, is increasingly sought after for its ease of installation, cost-effectiveness, and versatile capacity. However, the variability in environmental factors poses a significant challenge to photovoltaic (PV) power generation forecasting, which is crucial for maintaining power system stability and economic efficiency. In this paper, a novel muti-step-ahead PV power generation forecasting model by integrating single-step and multi-step forecasts from various time resolutions was developed. One-dimensional convolutional neural network (CNN) layers were used for single-step forecasting to capture specific temporal patterns, with the transformer model improving multi-step forecasting by leveraging the combined outputs of the CNN. This combination can provide accurate and immediate forecasts as well as the ability to identify longer-term generation trends. Using the DKASC-ASA-1A and 1B datasets for empirical validation, several preprocessing methods were applied and a series of experiments were conducted to compare the performance of the model with other widely used deep learning models. The framework proved to be capable of accurately predicting multi-step-ahead PV power generation at multiple time resolutions.

show abstract

Time-series anomaly detection with stacked Transformer representations and 1D convolutional network

Cited by 45 publications

References 10 publications

Diversified Adaptive Stock Selection Using Continual Graph Learning and Ensemble Approach

Diversified Adaptive Stock Selection Using Continual Graph Learning and Ensemble Approach

DCFF-MTAD: A Multivariate Time-Series Anomaly Detection Model Based on Dual-Channel Feature Fusion

A Multi-Step-Ahead Photovoltaic Power Forecasting Approach Using One-Dimensional Convolutional Neural Networks and Transformer

Contact Info

Product

Resources

About