DA-LSTM-VAE: Dual-Stage Attention-Based LSTM-VAE for KPI Anomaly Detection

Zhao, Yun; Zhang, Xiuguo; Shang, Zijing; Cao, Zhiying

doi:10.3390/e24111613

Cited by 6 publications

(5 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In other words, this action gives accurate predictions or evaluations of machine anomalies. Y. Zhao et al [25] introduced variational autoencoder (VAE) to time series anomaly detection. VAE delivers a powerful probabilistic modeling framework for time series data modeling and analysis.…”

Section: Literature Review and Background Materials 21 Literature Reviewmentioning

confidence: 99%

Ensemble Meta-Learning-Based Robust Chipping Prediction for Wafer Dicing

Chang,

Tsai,

2024

Electronics

View full text Add to dashboard Cite

Our previous study utilized importance analysis, random forest, and Barnes–Hut t-SNE dimensionality reduction to analyze critical dicing parameters and used bidirectional long short-term memory (BLSTM) to predict wafer chipping occurrence successfully in a single dicing machine. However, each dicing machine of the same type may produce unevenly distributed non-IID dicing signals, which may lead to the undesirable result that a pre-trained model trained by dicing machine #1 could not effectively predict chipping occurrence in dicing machine #2. Therefore, regarding the model robustness, this study introduces an ensemble meta-learning-based model that can evaluate many dicing machines for chipping prediction with high stability and accuracy. This approach constructs several base learners, such as the hidden Markov model (HMM), the variational autoencoder (VAE), and BLSTM, to form an ensemble learning. We use model-agnostic meta-learning (MAML) to train and test the ensemble learning model by several prediction tasks from machine #1. After MAML learning, we call the trained model a meta learner. Then, we successfully apply a retrieved data set from machine #2 to the meta learner for training and testing wafer chipping occurrence in this machine. As a result, our contribution to the robust chipping prediction on cross-machines can improve the yield of wafer dicing with a prediction accuracy of 93.21%, preserve the practical wearing of dicing kerfs, and significantly cut wafer manufacturing costs.

show abstract

Section: Literature Review and Background Materials 21 Literature Reviewmentioning

confidence: 99%

Ensemble Meta-Learning-Based Robust Chipping Prediction for Wafer Dicing

Chang,

Tsai,

2024

Electronics

View full text Add to dashboard Cite

show abstract

“…Mushtaq et al (Mushtaq et al, 2022) introduced a hybrid architecture that combines a deep LSTM autoencoder with bidirectional LSTM for classifying anomalies in intrusion detection. Zhao et al (Zhao et al, 2022) presented a dual attention network architecture that combines LSTM and VAE for analyzing time series data.…”

Section: Related Workmentioning

confidence: 99%

“…Autoencoder is a class of deep learning that solves the problem of dimensionality reduction for CCFD (Zhao et al, 2022). Studies reveal that LSTM autoencoders have the ability to deal with time series data as input for prediction when compared with regular autoencoders.…”

Section: Proposed Lstmae-xgb Frameworkmentioning

confidence: 99%

A combined framework based on LSTM autoencoder and XGBoost with adaptive threshold classification for credit card fraud detection

Prabha,

Priscilla

2024

The Scientific Temper

View full text Add to dashboard Cite

The digital invasion of the banking and financial sectors made life simple and easy. Traditional machine learning models have been studied in credit card fraud detection, but these models are often difficult to find effective for unseen patterns. This study proposes a combined framework of deep learning and machine learning models. The long short term memory autoencoder (LSTMAE) with attention mechanism is developed to extract high-level features and avoid overfitting of the model. The extracted features serve as input to the powerful ensemble model XGBoost to classify legitimate and fraudulent transactions. As the focus of fraud detection is to increase the recall rate, an adaptive threshold technique is proposed to estimate an optimal threshold value to enhance performance. The experiment was done with the IEEE-CIS fraud detection dataset available in Kaggle. The proposed model with optimal threshold has an increase in predicting fraudulent transactions. The research findings are compared with conventional ensemble techniques to find the generalization of the model. The proposed LSTMAE-XGB w/ attention method attained a good precision and recall of 94.2 and 90.5%, respectively, at the optimal threshold of θ = 0.22. The experimental results proved that the proposed approach is better at finding fraudulent transactions than other cutting-edge models

show abstract

“…This approach differs from the previously mentioned, since the goal is not to localize the position of the occurrence of an anomaly, but to detect if a built column, a MVTS, is anomalous as a whole. In contrast to approaches (e.g., [25]) which apply machine learning on KPI data, in this framework the KPI data is used as a separate mean for anomaly detection and for the unsupervised training set construction. The learning algorithm, however, is applied on the recorded time-series data.…”

Section: Machine Learning Based Anomaly Detectionmentioning

confidence: 99%

Detecting Anomalous Multivariate Time-Series via Hybrid Machine Learning

Terbuch

O’Leary

Khalili‐Motlagh‐Kasmaei

et al. 2023

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

This paper investigates the use of hybrid machine learning (HML) for the detection of anomalous multivariate time-series (MVTS). Focusing on a specific industrial use case from geotechnical engineering, where 100's of MVTS need to be analyzed and classified, has permitted extensive testing of the proposed methods with real measurement data. The novel hybrid anomaly detector combines two means for detection, creating redundancy and reducing the risk of missing defective elements in a safety relevant application. The two parts are: (1) anomaly detection based on approximately fifty physics-motivated key performance indicators (KPI) and ( 2) an unsupervised variational autoencoder with long short-term memory layers. The KPI capture expert knowledge on properties of the data that infer the quality of produced elements; these are used as a type of autolabeling. The goal of the extension using machine learning (ML) is to detect anomalies that the experts may not have foreseen. In contrast to anomaly detection in streaming data, where the goal is to locate an anomaly, each MVTS is complete in itself at the time of evaluation and is categorized as anomalous or non-anomalous.The paper compares the performance of different variationalautoencoder architectures (e.g., LSTM-VAE and BiLSTM-VAE). The results of using a genetic algorithm to optimize the hyperparameters of the different architectures are also presented. It is shown that modeling the industrial process as an assemblage of sub-processes yields a better discriminating power and permits the identification of inter-dependencies between the sub-processes. Interestingly, different autoencoder architectures may be optimal for different sub-processes; here two different architectures are combined to achieve superior performance. Extensive results are presented based on a very large set of real-time measurement data.

show abstract

DA-LSTM-VAE: Dual-Stage Attention-Based LSTM-VAE for KPI Anomaly Detection

Cited by 6 publications

References 61 publications

Ensemble Meta-Learning-Based Robust Chipping Prediction for Wafer Dicing

Ensemble Meta-Learning-Based Robust Chipping Prediction for Wafer Dicing

A combined framework based on LSTM autoencoder and XGBoost with adaptive threshold classification for credit card fraud detection

Detecting Anomalous Multivariate Time-Series via Hybrid Machine Learning

Contact Info

Product

Resources

About