The numerous recent breakthroughs in machine learning make imperative to carefully ponder how the scientific community can benefit from a technology that, although not necessarily new, is today living its golden age. This Grand Challenge review paper is focused on the present and future role of machine learning in Space Weather. The purpose is twofold. On one hand, we will discuss previous works that use machine learning for Space Weather forecasting, focusing in particular on the few areas that have seen most activity: the forecasting of geomagnetic indices, of relativistic electrons at geosynchronous orbits, of solar flares occurrence, of coronal mass ejection propagation time, and of solar wind speed. On the other hand, this paper serves as a gentle introduction to the field of machine learning tailored to the Space Weather community and as a pointer to a number of open challenges that we believe the community should undertake in the next decade. The recurring themes throughout the review are the need to shift our forecasting paradigm to a probabilistic approach focused on the reliable assessment of uncertainties, and the combination of physics-based and machine learning approaches, known as gray box. Plain Language SummaryIn the last decade, machine learning has achieved unforeseen results in industrial applications. In particular, the combination of massive data sets and computing with specialized processors (graphics processing units, or GPUs) can perform as well or better than humans in tasks like image classification and game playing. Space weather is a discipline that lives between academia and industry, given the relevant physical effects on satellites and power grids in a variety of applications, and the field therefore stands to benefit from the advances made in industrial applications. Today, machine learning poses both a challenge and an opportunity for the space weather community. The challenge is that the current data science revolution has not been fully embraced, possibly because space physicists remain skeptical of the gains achievable with machine learning. If the community can master the relevant technical skills, they should be able to appreciate what is possible within a few years time and what is possible within a decade. The clearest opportunity lies in creating space weather forecasting models that can respond in real time and that are built on both physics predictions and on observed data.

As a response to the Geospace Environment Modeling (GEM) “Global Radiation Belt Modeling Challenge,” a 3D diffusion model is used to simulate the radiation belt electron dynamics during two intervals of the Combined Release and Radiation Effects Satellite (CRRES) mission, 15 August to 15 October 1990 and 1 February to 31 July 1991. The 3D diffusion model, developed as part of the Dynamic Radiation Environment Assimilation Model (DREAM) project, includes radial, pitch angle, and momentum diffusion and mixed pitch angle‐momentum diffusion, which are driven by dynamic wave databases from the statistical CRRES wave data, including plasmaspheric hiss, lower‐band, and upper‐band chorus. By comparing the DREAM3D model outputs to the CRRES electron phase space density (PSD) data, we find that, with a data‐driven boundary condition at Lmax = 5.5, the electron enhancements can generally be explained by radial diffusion, though additional local heating from chorus waves is required. Because the PSD reductions are included in the boundary condition at Lmax = 5.5, our model captures the fast electron dropouts over a large L range, producing better model performance compared to previous published results. Plasmaspheric hiss produces electron losses inside the plasmasphere, but the model still sometimes overestimates the PSD there. Test simulations using reduced radial diffusion coefficients or increased pitch angle diffusion coefficients inside the plasmasphere suggest that better wave models and more realistic radial diffusion coefficients, both inside and outside the plasmasphere, are needed to improve the model performance. Statistically, the results show that, with the data‐driven outer boundary condition, including radial diffusion and plasmaspheric hiss is sufficient to model the electrons during geomagnetically quiet times, but to best capture the radiation belt variations during active times, pitch angle and momentum diffusion from chorus waves are required.

We report the properties of a novel type of sub-proton scale magnetic hole found in two dimensional particle-in-cell simulations of decaying turbulence with a guide field. The simulations were performed with a realistic value for ion to electron mass ratio. These structures, electron vortex magnetic holes (EVMHs), have circular cross-section. The magnetic field depression is associated with a diamagnetic azimuthal current provided by a population of trapped electrons in petal-like orbits. The trapped electron population provides a mean azimuthal velocity and since trapping preferentially selects high pitch angles, a perpendicular temperature anisotropy. The structures arise out of initial perturbations in the course of the turbulent evolution of the plasma, and are stable over at least 100 electron gyroperiods. We have verified the model for the EVMH by carrying out test particle and PIC simulations of isolated structures in a uniform plasma. It is found that (quasi-)stable structures can be formed provided that there is some initial perpendicular temperature anisotropy at the structure location. The properties of these structures (scale size, trapped population, etc.) are able to explain the observed properties of magnetic holes in the terrestrial plasma sheet. EVMHs may also contribute to turbulence properties, such as intermittency, at short scale lengths in other astrophysical plasmas. V C 2015 AIP Publishing LLC.

[1] The kinetic electron firehose instability (EFI) is thought to be a crucial mechanism for constraining the observed electron anisotropy in expanding astrophysical plasmas, such as the solar wind. The EFI arises in a bi-Maxwellian plasma when the parallel temperature is greater than the perpendicular one, and its effect is to reduce anisotropy. We study this mechanism via kinetic linear theory, extending and refining previous results, and by new two-dimensional particle-in-cell (PIC) simulations with physical mass ratio. The results of PIC simulations show under which conditions the EFI can indeed be regarded as a constraint for electron distribution function. The detailed electron physics near marginal stability condition is discussed, with emphasis on the competition between growing and damping modes and on wave patterns formed at the nonlinear stage. The results also suggest an observational signature that the EFI has operated, namely the appearance of low-frequency, quasiperpendicular whistler/electron-cyclotron waves.

The solar wind‐magnetosphere system is nonlinear. The solar wind drivers of geosynchronous electrons with energy range of 1.8–3.5 MeV are investigated using mutual information, conditional mutual information (CMI), and transfer entropy (TE). These information theoretical tools can establish linear and nonlinear relationships as well as information transfer. The information transfer from solar wind velocity (Vsw) to geosynchronous MeV electron flux (Je) peaks with a lag time of 2 days. As previously reported, Je is anticorrelated with solar wind density (nsw) with a lag of 1 day. However, this lag time and anticorrelation can be attributed at least partly to the Je(t + 2 days) correlation with Vsw(t) and nsw(t + 1 day) anticorrelation with Vsw(t). Analyses of solar wind driving of the magnetosphere need to consider the large lag times, up to 3 days, in the (Vsw, nsw) anticorrelation. Using CMI to remove the effects of Vsw, the response of Je to nsw is 30% smaller and has a lag time < 24 h, suggesting that the MeV electron loss mechanism due to nsw or solar wind dynamic pressure has to start operating in < 24 h. nsw transfers about 36% as much information as Vsw (the primary driver) to Je. Nonstationarity in the system dynamics is investigated using windowed TE. When the data are ordered according to transfer entropy value, it is possible to understand details of the triangle distribution that has been identified between Je(t + 2 days) versus Vsw(t).

In this study, we present a method that combines a Long Short-Term Memory (LSTM) recurrent neural network with a Gaussian process (GP) model to provide up to 6-hr-ahead probabilistic forecasts of the Dst geomagnetic index. The proposed approach brings together the sequence modeling capabilities of a recurrent neural network with the error bars and confidence bounds provided by a GP. Our model is trained using the hourly OMNI and Global Positioning System (GPS) databases, both of which are publicly available. We first develop a LSTM network to get a single-point prediction of Dst. This model yields great accuracy in forecasting the Dst index from 1 to 6 hr ahead, with a correlation coefficient always higher than 0.873 and a root-mean-square error lower than 9.86. However, even if global metrics show excellent performance, it remains poor in predicting intense storms (Dst < À250 nT) 6 hr in advance. To improve it and to obtain probabilistic forecasts, we combine the LSTM model obtained with a GP and evaluate the hybrid predictor using the receiver operating characteristic curve and the reliability diagram. We conclude that this hybrid methodology provides improvements in the forecast of geomagnetic storms, from 1 to 6 hr ahead.

We present a four‐category classification algorithm for the solar wind, based on Gaussian Process. The four categories are the ones previously adopted in Xu and Borovsky (2015): ejecta, coronal hole origin plasma, streamer belt origin plasma, and sector reversal origin plasma. The algorithm is trained and tested on a labeled portion of the OMNI data set. It uses seven inputs: the solar wind speed Vsw, the temperature standard deviation σT, the sunspot number R, the F10.7 index, the Alfven speed vA, the proton specific entropy Sp, and the proton temperature Tp compared to a velocity‐dependent expected temperature. The output of the Gaussian Process classifier is a four‐element vector containing the probabilities that an event (one reading from the hourly averaged OMNI database) belongs to each category. The probabilistic nature of the prediction allows for a more informative and flexible interpretation of the results, for instance, being able to classify events as “undecided.” The new method has a median accuracy larger than 90% for all categories, even using a small set of data for training. The Receiver Operating Characteristic curve and the reliability diagram also demonstrate the excellent quality of this new method. Finally, we use the algorithm to classify a large portion of the OMNI data set, and we present for the first time transition probabilities between different solar wind categories. Such probabilities represent the “climatological” statistics that determine the solar wind baseline.

a b s t r a c tWe describe a spectral method for the numerical solution of the Vlasov-Poisson system where the velocity space is decomposed by means of an Hermite basis, and the configuration space is discretized via a Fourier decomposition. The novelty of our approach is an implicit time discretization that allows exact conservation of charge, momentum and energy. The computational efficiency and the cost-effectiveness of this method are compared to the fully-implicit PIC method recently introduced by Lapenta (2011) andChen et al. (2011). The following examples are discussed: Langmuir wave, Landau damping, ion-acoustic wave, two-stream instability. The Fourier-Hermite spectral method can achieve solutions that are several orders of magnitude more accurate at a fraction of the cost with respect to PIC.