Recurrent Neural Networks for Online Remaining Useful Life Estimation in Ion Mill Etching System

Abstract

We describe the approach – submitted as part of the 2018 PHM Data Challenge – for estimating time-to-failure or Remaining Useful Life (RUL) of Ion Mill Etching Systems in an online fashion using data from multiple sensors. RUL estimation from multi-sensor data can be considered as learning a regression function that maps a multivariate time series to a real-valued number, i.e. the RUL. We use a deep Recurrent Neural Network (RNN) to learn the metric regression function from multivariate time series. We highlight practical aspects of the RUL estimation problem in this data challenge such as i) multiple operating conditions, ii) lack of knowledge of exact onset of failure or degradation, iii) different operational behavior across tools in terms of range of values of parameters, etc. We describe our solution in the context of these challenges. Importantly, multiple modes of failure are possible in an ion mill etching system; therefore, it is desirable to estimate the RUL with respect to each of the failure modes. The data challenge considers three such modes of failures and requires estimating RULs with respect to each one, implying learning three metric regression functions - one corresponding to each failure mode. We propose a simple yet effective extension to existing methods of RUL estimation using RNN based regression to learn a single deep RNN model that can simultaneously estimate RULs corresponding to all three failure modes. Our best model is an ensemble of two such RNN models and achieves a score of 1:91 X 10^7 on the final validation set..

A dynamic mode decomposition based deep learning technique for prognostics

“…Some of the papers applying LSTM share a few aspects such as datasets and machine components as mentioned earlier in this section. Other common grounds exist amongst LSTM based prognostics papers include the use of the dropout technique [11], [13], [15], [17], [19], [23][24][25], [27][28][29][30] In addition, the dropout technique can also benefit another variation of RNN that is the gated recurrent unit [32]. Dropout is helpful for model regularization and to prevent overfitting.…”

“…Early stopping was also employed to stop the training when no significant improvement was achieved on the validation dataset. In [17], ordinal regression (OR) was used with the LSTM network for the purposes of RUL estimation. Data augmentation based oversampling was used to overcome the shortage of labeled data.…”

“…Another solution to RUL prediction uncertainty is to use a bootstrap of the data inputted into the LSTM networks [15] or an ensemble [29] of LSTM networks. Bootstrapping helps reducing uncertainty by sampling with replacement.…”

“…It is worth mentioning that ensemble methods can be implemented by bootstrapping the input data going into the LSTM network or by using different LSTM network topologies. In [29], different LSTM network topologies were implemented in the ensemble. The best performing network was then applied to individual sub-datasets of the NASA turbofan engine dataset.…”

“…As mentioned earlier, LSTM network is a powerful prognostic tool. It has been used to predict RUL for a variety of components and systems such as aero-propulsion systems [11][12][13][14][15][16][17][18][19][20][21], mechanical machinery components [22], [23] [24] batteries [25][26][27], and milling systems [13], [21] [ [28][29][30]. In addition, the original RNN architecture and an RNN variation named gated recurrent unit (GRU) were implemented on aero-propulsion system in [31] and [32], respectively.…”