Comparative analysis of machine learning models for solar flare prediction

Abstract

In this paper, we develop five machine learning models, neural network (NN), long short-term memory (LSTM), LSTM based on attention mechanism (LSTM-A), bidirectional LSTM (BLSTM), and BLSTM based on attention mechanism (BLSTM-A), for predicting whether a flare event of ≥C class or ≥M class will occur within 24 hr. We use solar active region samples provided by the Space-weather Helioseismic and Magnetic Imager Active Region Patches data from May 2010 to September 2018. The samples are divided into four categories (No-flare, C, M, and X) and generate 10 separate data sets. Then, after training, validating, and testing our models, we compare the results with the true skill statistics (TSS) as the assessment metric. The main results are as follows. (1) The TSS scores of ≥C class are 0.5472 ± 0.0809, 0.6425 ± 0.0685, 0.6904 ± 0.0575, 0.6681 ± 0.0573, and 0.6833 ± 0.0531 for NN, LSTM, BLSTM, LSTM-A and BLSTM-A, respectively. The TSS scores of ≥M class are 0.5723 ± 0.1139, 0.6579 ± 0.0758, 0.5943 ± 0.0712, 0.6493 ± 0.0826, and 0.5932 ± 0.0723, respectively. (2) For the first time, we add an attention mechanism to BLSTM for flare prediction, which improves the performance of the model for ≥C class. (3) Among the five models, the prediction model based on deep learning algorithms is generally superior to the model based on the traditional machine learning algorithm. The performance of the LSTM models is comparable to that of the BLSTM models. In general, LSTM-A for ≥C class performs better than other models. In addition, We also discuss the influence of 10 features on LSTM-A, and we find that removing the least significant feature will result in better performance than using all 10 features together, and the TSS score of the model will improve to 0.7059 ± 0.0440.