The Challenge of Machine Learning in Space Weather: Nowcasting and Forecasting

Abstract

The numerous recent breakthroughs in machine learning make imperative to carefully ponder how the scientific community can benefit from a technology that, although not necessarily new, is today living its golden age. This Grand Challenge review paper is focused on the present and future role of machine learning in Space Weather. The purpose is twofold. On one hand, we will discuss previous works that use machine learning for Space Weather forecasting, focusing in particular on the few areas that have seen most activity: the forecasting of geomagnetic indices, of relativistic electrons at geosynchronous orbits, of solar flares occurrence, of coronal mass ejection propagation time, and of solar wind speed. On the other hand, this paper serves as a gentle introduction to the field of machine learning tailored to the Space Weather community and as a pointer to a number of open challenges that we believe the community should undertake in the next decade. The recurring themes throughout the review are the need to shift our forecasting paradigm to a probabilistic approach focused on the reliable assessment of uncertainties, and the combination of physics-based and machine learning approaches, known as gray box. Plain Language SummaryIn the last decade, machine learning has achieved unforeseen results in industrial applications. In particular, the combination of massive data sets and computing with specialized processors (graphics processing units, or GPUs) can perform as well or better than humans in tasks like image classification and game playing. Space weather is a discipline that lives between academia and industry, given the relevant physical effects on satellites and power grids in a variety of applications, and the field therefore stands to benefit from the advances made in industrial applications. Today, machine learning poses both a challenge and an opportunity for the space weather community. The challenge is that the current data science revolution has not been fully embraced, possibly because space physicists remain skeptical of the gains achievable with machine learning. If the community can master the relevant technical skills, they should be able to appreciate what is possible within a few years time and what is possible within a decade. The clearest opportunity lies in creating space weather forecasting models that can respond in real time and that are built on both physics predictions and on observed data.

Leveraging the mathematics of shape for solar magnetic eruption prediction

“…In the past decade, the large increase in magnetogram data afforded by space missions and advances in data access have shifted the forefront of flare-prediction research from empirical modeling methods to "data analytic" approaches such as machine learning. Camporeale (2019) summarizes the state of the art in machine learning approaches to space weather applications. In ML-based prediction applications, characteristic "features" of the photospheric magnetic field, sometimes combined with features seen in simultaneous Extreme UltraViolet (EUV) images of the solar corona, are used in a statistical sense to "train" a computational model to predict the probability of an eruption within a given time period (usually 24 hours).…”

Forecasting Megaelectron‐Volt Electrons Inside Earth's Outer Radiation Belt: PreMevE 2.0 Based on Supervised Machine Learning Algorithms

“…As explained by Camporeale (), supervised regressors try to find the mapping relationship between a set of multidimensional inputs x = ( x 1 , x 2 , …, x N ) and its corresponding scalar output label y , under the general form …”

Challenges and Opportunities in Magnetospheric Space Weather Prediction

“…For example, the terms such as skill, accuracy, and reliability all have precise meanings across the weather and climate research communities. For clear definitions and explanations of this terminology, as well as validation methodologies, we direct the reader to textbooks such at those by Wilks (1995) and Jolliffe and Stephenson (2011), as well as some space weather domain literature (Camporeale, 2019;Morley, Brito, & Welling, 2018;M. W. Liemohn et al, 2018).…”