A Novel Ensemble Forecasting Algorithm based on Distributed Deep Learning Network

Abstract

This paper proposes an ensemble model based on distribution deep learning network. The ensemble model is composed of deep belief network (DBN) for reconstructing original data, and the bidirectional long short-term memory (BLSTM) method is used for prediction due to its good results in big data applications. The dynamic weighting strategies are proposed and applied to the sub models of the ensemble by a weighted least square method. The weight update with variable training sets and the predictions for each ensemble are obtained from the distributed computing engine Apache Spark. The performance of the proposed model is evaluated on wind data on the wind farm of the Hexi Corridor in China. The simulation results show that the dynamic ensemble algorithm performs well, which is a very valuable result for the forecasting of big data time series. Furthermore, the results are successfully compared with back propagation neural Network (BPNN), LSTM, BLSTM, and stacked LSTMs with memory between batches (SBLSTM), improving the accuracy of prediction.