Optimizing long short-term memory recurrent neural networks using ant colony optimization to predict turbine engine vibration

Abstract

This article expands on research that has been done to develop a recurrent neural network (RNN) capable of predicting aircraft engine vibrations using long short-term memory (LSTM) neurons. LSTM RNNs can provide a more generalizable and robust method for prediction over analytical calculations of engine vibration, as analytical calculations must be solved iteratively based on specific empirical engine parameters, making this approach ungeneralizable across multiple engines. In initial work, multiple LSTM RNN architectures were proposed, evaluated and compared. This research improves the performance of the most effective LSTM network design proposed in the previous work by using a promising neuroevolution method based on ant colony optimization (ACO) to develop and enhance the LSTM cell structure of the network. A parallelized version of the ACO neuroevolution algorithm has been developed and the evolved LSTM RNNs were compared to the previously used fixed topology. The evolved networks were trained on a large database of flight data records obtained from an airline containing flights that suffered from excessive vibration. Results were obtained using MPI (Message Passing Interface) on a high performance computing (HPC) cluster, evolving 1000 different LSTM cell structures using 168 cores over 4 days. The new evolved LSTM cells showed an improvement of 1.35%, reducing prediction error from 5.51% to 4.17% when predicting excessive engine vibrations 10 seconds in the future, while at the same time dramatically reducing the number of weights from 21,170 to 11,810.

Investigating recurrent neural network memory structures using neuro-evolution

“…SGD was run with a learning rate η = 0.001, utilizing Nesterov momentum with mu = 0.9. No dropout regularization was used since it has been shown in other work to reduce performance when training RNNs for time series prediction [24]. To prevent exploding gradients, gradient clipping (as described by Pascanu et al [35]) was used when the norm of the gradient was above a threshold of 1.0.…”

A long-term prediction approach based on long short-term memory neural networks with automatic parameter optimization by Tree-structured Parzen Estimator and applied to time-series data of NPP steam generators

“…RNN is a network with feedback connections from the hidden and output layers to the preceding ones, by which the dynamics of sequential data can be captured and the memories of the previous patterns are retained via cycles in the network. In the last decade, RNNs have been extensively investigated for a variety of prognostic applications, including engine systems [20]- [23], lithium-ion batteries [24]- [26], rolling element bearings [27]- [30] and fuel cells [31], [32]. Zhang et al [24] utilized a RNN to extract the long-term dependencies underlying in the battery capacity degradation process.…”

Ship Motion Attitude Prediction Based on an Adaptive Dynamic Particle Swarm Optimization Algorithm and Bidirectional LSTM Neural Network

“…However, BiLSTM does not solve the problem of random initialization of neural network parameters, which will affect the nonlinear learning ability. In view of this shortcoming, the existing methods to optimize the neural network are mainly to improve the error function and excitation function [17]- [19], or to optimize the initial parameters of the network by using intelligent algorithms [20]- [23]. The current intelligent algorithms for parameter optimization mainly include particle swarm optimization (PSO) algorithm [24] and ant colony optimization algorithm, but these algorithms are often prone to the problem of local optimal, which leads to the failure to find the global optimal solution in the process of parameter optimization.…”