Remaining Useful Life Prediction Using Temporal Convolution with Attention

Abstract

Prognostic techniques attempt to predict the Remaining Useful Life (RUL) of a subsystem or a component. Such techniques often use sensor data which are periodically measured and recorded into a time series data set. Such multivariate data sets form complex and non-linear inter-dependencies through recorded time steps and between sensors. Many current existing algorithms for prognostic purposes starts to explore Deep Neural Network (DNN) and its effectiveness in the field. Although Deep Learning (DL) techniques outperform the traditional prognostic algorithms, the networks are generally complex to deploy or train. This paper proposes a Multi-variable Time Series (MTS) focused approach to prognostics that implements a lightweight Convolutional Neural Network (CNN) with attention mechanism. The convolution filters work to extract the abstract temporal patterns from the multiple time series, while the attention mechanisms review the information across the time axis and select the relevant information. The results suggest that the proposed method not only produces a superior accuracy of RUL estimation but it also trains many folds faster than the reported works. The superiority of deploying the network is also demonstrated on a lightweight hardware platform by not just being much compact, but also more efficient for the resource restricted environment.

Shared Temporal Attention Transformer for Remaining Useful Lifetime Estimation

“…In this section, we evaluate the sequence RUL predictor models presented in this paper against the literature model performances based on their generated scores for the C-MAPSS Turbofan Engine dataset. As evident in Table 8, the proposed STAT and FeaR-STAT model outperforms the best literature score [18] for all the sub-datasets, with the exception of FD001 for the FeaR-STAT model. The STAT model strikes a fine balance throughout the sub-datasets.…”

“…This unit width kernel allows for sharing kernel weights across raw sensors and enhancing the network's ability to learn abstract feature information. An attentionbased CNN approach is proposed in [18] where the CNN filters extract features across multiple temporal axes, which are further analyzed by the attention layer before generating the RUL. The attention mechanism in [18] replaces the Softmax activation with a Sigmoid activation, intending to use additional multivariate features for estimating RUL.…”

“…An attentionbased CNN approach is proposed in [18] where the CNN filters extract features across multiple temporal axes, which are further analyzed by the attention layer before generating the RUL. The attention mechanism in [18] replaces the Softmax activation with a Sigmoid activation, intending to use additional multivariate features for estimating RUL. The attention-based stacked LSTM RUL predictor proposed in [19] implements the global Luong concatenation method to calculate attention alignment scores to predict RULs for the C-MAPSS dataset.…”

“…The RMSE for all methods with the attention mechanism which perform similarly to the STAT and FeaR-STAT with respect to the scores are shown in Table 9. The proposed STAT and FeaR-STAT transformers fall slightly behind the Transformer-encoder with Gated CNN model [29], DAG [23] and the CNN + Attention [18] for the FD001 RMSE. However, proposed transformer RMSEs outperform in all other operating conditions compared to existing literature underlining the capability of the approaches.…”

Deep learning-based anomaly-onset aware remaining useful life estimation of bearings

“…Complex models may be learned using deep learning algorithms using large amounts of training data. Deep learning algorithms widely used in RUL estimation include autoencoders ( Xu et al, 2021 ), convolutional neural networks (CNN), CNN with attention mechanism ( Tan & Teo, 2021 ; Wang et al, 2021 ), long-short term memory ( Xia et al, 2021 ), gated recurrent unit ( She & Jia, 2021 ), random forest ( Chen et al, 2020 ) and support vector machines ( Xue et al, 2020 ).…”