Long Short-Term Memory Neural Network with Transfer Learning and Ensemble Learning for Remaining Useful Life Prediction

Wang, Lixiong; Liu, Hanjie; Zhang, Pan; Fan, Dian; Zhou, Ciming; Wang, Zhigang

doi:10.3390/s22155744

Cited by 16 publications

(11 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It means that the classifiers which have better performance are assigned larger weights by minimizing the loss function. The final ensemble prediction is obtained by averaging from the optimal weights of classifiers [84]. In other words, it is possible to calculate the optimal weights of the network by using several fullyconnected layers and then ensemble them to achieve an optimal ensemble classifier [51].…”

Section: B Ensemble Learningmentioning

confidence: 99%

ERDeR: The Combination of Statistical Shrinkage Methods and Ensemble Approaches to Improve the Performance of Deep Regression

Farhadi,

Feizi-Derakhshi,

Bevrani

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Ensembling is a powerful technique to obtain the most accurate results. In some cases, the large number of learners in ensemble learning mostly increases both computational load during the test phase and error rate. To solve this problem, in this paper we propose an Ensemble of Reduced Deep Regression (ERDeR) model, which is a combination of Deep Regressions (DRs), shrinkage methods, and ensemble approaches. The framework of the proposed model contains three phases. The first phase includes base regressions in which parallel DRs are used as learners. The role of these DRs is to extract features of input data and make prediction. In the second phase, to automatically reduce and select the most suitable DRs, shrinkage methods such as Least Absolute Shrinkage and Selection Operator (LASSO) and Elastic Net (EN) are employed. These models are compared with the non-shrinkage model. The last phase is ensemble phase, which consists of three different ensemble methods namely Multi-Layer Perceptron (MLP), Weighted Average (WA), and Simple Average (SA). These ensemble methods are used to aggregate the remaining learners from previous steps. Finally, the proposed model is applied to Monte Carlo simulation data and three real datasets including Boston House Price, Real Estate Valuation and Gold Price per Ounce. The results show that after applying the shrinkage methods the error rate is significantly reduced and the model accuracy is increased. Accordingly, the results of combining shrinkage methods and ensemble approaches not only decreased the computational load during test phase, but also increased the model accuracy.

show abstract

Section: B Ensemble Learningmentioning

confidence: 99%

ERDeR: The Combination of Statistical Shrinkage Methods and Ensemble Approaches to Improve the Performance of Deep Regression

Farhadi,

Feizi-Derakhshi,

Bevrani

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…It can remember the relationship of the current information with the long-term information in the time sequence. The hidden level of traditional RNN has been replaced with memory cell (Wang et al 2022a). It is comprised of a forget gate, input, and output gate, as shown in Fig.…”

Section: Long Short-term Memory (Lstm)mentioning

confidence: 99%

“…It is comprised of a forget gate, input, and output gate, as shown in Fig. 1 (Liu et al 2020;Wang et al 2022a).…”

Section: Long Short-term Memory (Lstm)mentioning

confidence: 99%

Optimizing long-short-term memory models via metaheuristics for decomposition aided wind energy generation forecasting

Pavlov-Kagadejev,

Jovanovic,

Bacanin

et al. 2024

Artif Intell Rev

View full text Add to dashboard Cite

Power supply from renewable energy is an important part of modern power grids. Robust methods for predicting production are required to balance production and demand to avoid losses. This study proposed an approach that incorporates signal decomposition techniques with Long Short-Term Memory (LSTM) neural networks tuned via a modified metaheuristic algorithm used for wind power generation forecasting. LSTM networks perform notably well when addressing time-series prediction, and further hyperparameter tuning by a modified version of the reptile search algorithm (RSA) can help improve performance. The modified RSA was first evaluated against standard CEC2019 benchmark instances before being applied to the practical challenge. The proposed tuned LSTM model has been tested against two wind production datasets with hourly resolutions. The predictions were executed without and with decomposition for one, two, and three steps ahead. Simulation outcomes have been compared to LSTM networks tuned by other cutting-edge metaheuristics. It was observed that the introduced methodology notably exceed other contenders, as was later confirmed by the statistical analysis. Finally, this study also provides interpretations of the best-performing models on both observed datasets, accompanied by the analysis of the importance and impact each feature has on the predictions.

show abstract

“…On one hand, ML-based aero-engine RUL prediction is being studied by combining different neural networks to construct hybrid models or by using the most recent algorithmic models to improve prediction accuracy, such as Li et al [ 32 ], Liu et al [ 33 ], Zhao et al [ 34 ], and others. On the other hand, because of the multi-serial and time-series nature of the aero-engine dataset, many studies, such as Jiang et al [ 35 ], Wang et al [ 36 ], and Zhang et al [ 18 ], have attempted to make the model more focused on important information and achieve long time series prediction by introducing SA and transformer structures. As a result, a lot of effort is spent on building complex ML models, and the domain knowledge of the devices is neglected and not fully utilized.…”

Section: Introductionmentioning

confidence: 99%

Multiform Informed Machine Learning Based on Piecewise and Weibull for Engine Remaining Useful Life Prediction

Zhou

Yao

Liu

et al. 2023

Sensors

View full text Add to dashboard Cite

Informed machine learning (IML), which strengthens machine learning (ML) models by incorporating external knowledge, can get around issues like prediction outputs that do not follow natural laws and models, hitting optimization limits. It is therefore of significant importance to investigate how domain knowledge of equipment degradation or failure can be incorporated into machine learning models to achieve more accurate and more interpretable predictions of the remaining useful life (RUL) of equipment. Based on the informed machine learning process, the model proposed in this paper is divided into the following three steps: (1) determine the sources of the two types of knowledge based on the device domain knowledge, (2) express the two forms of knowledge formally in Piecewise and Weibull, respectively, and (3) select different ways of integrating them into the machine learning pipeline based on the results of the formal expression of the two types of knowledge in the previous step. The experimental results show that the model has a simpler and more general structure than existing machine learning models and that it has higher accuracy and more stable performance in most datasets, particularly those with complex operational conditions, which demonstrates the effectiveness of the method in this paper on the C-MAPSS dataset and assists scholars in properly using domain knowledge to deal with the problem of insufficient training data.

show abstract

Long Short-Term Memory Neural Network with Transfer Learning and Ensemble Learning for Remaining Useful Life Prediction

Cited by 16 publications

References 24 publications

ERDeR: The Combination of Statistical Shrinkage Methods and Ensemble Approaches to Improve the Performance of Deep Regression

ERDeR: The Combination of Statistical Shrinkage Methods and Ensemble Approaches to Improve the Performance of Deep Regression

Optimizing long-short-term memory models via metaheuristics for decomposition aided wind energy generation forecasting

Multiform Informed Machine Learning Based on Piecewise and Weibull for Engine Remaining Useful Life Prediction

Contact Info

Product

Resources

About