How to Train Your Flare Prediction Model: Revisiting Robust Sampling of Rare Events

Abstract

We present a case study of solar flare forecasting by means of metadata feature time series, by treating it as a prominent class-imbalance and temporally coherent problem. Taking full advantage of pre-flare time series in solar active regions is made possible via the Space Weather Analytics for Solar Flares (SWAN-SF) benchmark data set, a partitioned collection of multivariate time series of active region properties comprising 4075 regions and spanning over 9 yr of the Solar Dynamics Observatory period of operations. We showcase the general concept of temporal coherence triggered by the demand of continuity in time series forecasting and show that lack of proper understanding of this effect may spuriously enhance models’ performance. We further address another well-known challenge in rare-event prediction, namely, the class-imbalance issue. The SWAN-SF is an appropriate data set for this, with a 60:1 imbalance ratio for GOES M- and X-class flares and an 800:1 imbalance ratio for X-class flares against flare-quiet instances. We revisit the main remedies for these challenges and present several experiments to illustrate the exact impact that each of these remedies may have on performance. Moreover, we acknowledge that some basic data manipulation tasks such as data normalization and cross validation may also impact the performance; we discuss these problems as well. In this framework we also review the primary advantages and disadvantages of using true skill statistic and Heidke skill score, two widely used performance verification metrics for the flare-forecasting task. In conclusion, we show and advocate for the benefits of time series versus point-in-time forecasting, provided that the above challenges are measurably and quantitatively addressed.

Prediction of Solar Proton Events with Machine Learning: Comparison with Operational Forecasts and "All-Clear" Perspectives

“…As one can see, there are approximately twice as many samples in the train data set than in the test data set, and the class-imbalance ratios in both data sets are also kept approximately equal to 1 to 34. Because the separation is done by keeping long chunks in the same data sets, we avoid the temporal coherence problem (Ahmadzadeh et al 2021).…”

“…Because the data set suffers from class-imbalance, we adopt a strategy to mitigate this (Ahmadzadeh et al 2021). In this work, we oversample the positive cases (days with SPEs) in the dataset to balance the number of negative cases.…”

“…This ratio (defined here as a class-imbalance) becomes even stronger for higher fluxes or energy thresholds. Difficulties and strategies for forecasting space weather events for highly-imbalanced data sets were pointed out by Ahmadzadeh et al (2021), and a need for balanced data sets for space weather prediction was emphasized by (Martens & Angryk 2018). In addition, SPEs may be significantly delayed (up to tens of hours) with respect to the onset time of the preceding solar flare.…”

Using Multiple Instance Learning for Explainable Solar Flare Prediction

“…The prediction of solar flares is an active topic in the space weather community (Ahmed et al, 2013;Bobra and Couvidat, 2015;Boucheron et al, 2015;Nishizuka et al, 2017;Liu et al, 2017;Jonas et al, 2018;Florios et al, 2018;Huang et al, 2018;Liu et al, 2019;Armstrong and Fletcher, 2019;Galvez et al, 2019;Chen et al, 2019;Ahmadzadeh et al, 2021). To this end, solar data recorded in different wavelenghts and spatio-temporal resolutions is gathered by a fleet of satellites and an array of telescopes.…”

On the differences in the periodic behaviour of magnetic helicity flux in flaring active regions with and without X-class events

“…for the functioning of electric power grids, aviation, radio communication, GPS, and spacebased facilities (Eastwood et al 2017). For this reason, it is vital to further develop existing prediction capabilities through the identification of observable precur-sors of flares and CMEs (see Barnes et al 2016;Leka et al 2019;Kusano et al 2020;Patsourakos et al 2020;Ahmadzadeh et al 2021;Georgoulis, Manolis K. et al 2021). Understanding the physical processes of flare and CME precursors is still a challenging task in solar physics research (Florios et al 2018;Korsós et al 2019, and in their references).…”