Geomagnetic Index <i>Kp</i> Forecasting With LSTM

Tan, Yao; Hu, Qinghua; Wang, Zhen; Zhong, Qing

doi:10.1002/2017sw001764

Cited by 59 publications

(64 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A number of different attempts have been proposed previously to mitigate the imbalance in the Kp probability distribution function. For instance, Tan et al () built two different models for high and low activities, and a third model was used to choose which of the models to employ for forecasts. Another possible approach is to use a target‐dependent cost function during the training of the neural network or any other ML algorithm.…”

Section: Resultsmentioning

confidence: 99%

“…Due to the high cadence of the input solar wind variables, following Wintoft et al () and Tan et al (), we extract statistical information out of these variables before using them as inputs. Specifically, we consider 3‐hr windows of measurements for which we calculate average values as well as minimum and maximum values.…”

Section: Model Inputsmentioning

confidence: 99%

“…They also showed how the performance of the method can be improved by using the near‐real‐time values of the Kp index derived from near‐real‐time ground observations. The application of various ML tools has been the subject of a number of recent studies (Ayala Solares et al, ; Bala & Reiff, ; Ji et al, ; Tan et al, ; Wang et al, ; Wintoft et al, ).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Nowcasting and Predicting the Kp Index Using Historical Values and Real‐Time Observations

2019

View full text Add to dashboard Cite

Current algorithms for the real-time prediction of the Kp index use a combination of models empirically driven by solar wind measurements at the L1 Lagrange point and historical values of the index. In this study, we explore the limitations of this approach, examining the forecast for short and long lead times using measurements at L1 and Kp time series as input to artificial neural networks. We explore the relative efficiency of the solar wind-based predictions, predictions based on recurrence, and predictions based on persistence. Our modeling results show that for short-term forecasts of approximately half a day, the addition of the historical values of Kp to the measured solar wind values provides a barely noticeable improvement. For a longer-term forecast of more than 2 days, predictions can be made using recurrence only, while solar wind measurements provide very little improvement for a forecast with long horizon times. We also examine predictions for disturbed and quiet geomagnetic activity conditions. Our results show that the paucity of historical measurements of the solar wind for high Kp results in a lower accuracy of predictions during disturbed conditions. Rebalancing of input data can help tailor the predictions for more disturbed conditions.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Model Inputsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Nowcasting and Predicting the Kp Index Using Historical Values and Real‐Time Observations

2019

View full text Add to dashboard Cite

show abstract

“…The use of a NN to forecast Kp either one or multiple hours in advance has been proposed in Bala et al (), Boberg et al (), Costello (), Gholipour et al (), Tan et al (), Uwamahoro and Habarulema (), Valach and Prigancová (), Wing et al (), and Wintoft et al (), among others. Real‐time forecasts based on some of these models are running at RWC, Sweden (http://www.lund.irf.se/forecast/kp/), Rice Space Institute, USA (http://mms.rice.edu/mms/forecast.php), INPE, Brazil (http://www2.inpe.br/climaespacial/portal/swd-forecast/), and the Space Environment Prediction Center, China (http://eng.sepc.ac.cn/Kp3HPred.php).…”

Section: Review Of Machine Learning In Space Weathermentioning

confidence: 99%

The Challenge of Machine Learning in Space Weather: Nowcasting and Forecasting

2019

View full text Add to dashboard Cite

The numerous recent breakthroughs in machine learning make imperative to carefully ponder how the scientific community can benefit from a technology that, although not necessarily new, is today living its golden age. This Grand Challenge review paper is focused on the present and future role of machine learning in Space Weather. The purpose is twofold. On one hand, we will discuss previous works that use machine learning for Space Weather forecasting, focusing in particular on the few areas that have seen most activity: the forecasting of geomagnetic indices, of relativistic electrons at geosynchronous orbits, of solar flares occurrence, of coronal mass ejection propagation time, and of solar wind speed. On the other hand, this paper serves as a gentle introduction to the field of machine learning tailored to the Space Weather community and as a pointer to a number of open challenges that we believe the community should undertake in the next decade. The recurring themes throughout the review are the need to shift our forecasting paradigm to a probabilistic approach focused on the reliable assessment of uncertainties, and the combination of physics-based and machine learning approaches, known as gray box. Plain Language SummaryIn the last decade, machine learning has achieved unforeseen results in industrial applications. In particular, the combination of massive data sets and computing with specialized processors (graphics processing units, or GPUs) can perform as well or better than humans in tasks like image classification and game playing. Space weather is a discipline that lives between academia and industry, given the relevant physical effects on satellites and power grids in a variety of applications, and the field therefore stands to benefit from the advances made in industrial applications. Today, machine learning poses both a challenge and an opportunity for the space weather community. The challenge is that the current data science revolution has not been fully embraced, possibly because space physicists remain skeptical of the gains achievable with machine learning. If the community can master the relevant technical skills, they should be able to appreciate what is possible within a few years time and what is possible within a decade. The clearest opportunity lies in creating space weather forecasting models that can respond in real time and that are built on both physics predictions and on observed data.

show abstract

“…Yeh et al (2017) used DCGAN to perform image completion for damaged images and achieved impressive results. Although the utility of conventional AI algorithms in space physics has been investigated for decades (e.g., Oyeyemi et al, 2006;Poole & McKinnell, 2000;Weigel et al, 1999;Wintoft & Cander, 1999), deep learning has yet attracted a broad interest except for a handful of studies on forecasting the solar flare (Hada-Muranushi et al, 2016), the disturbance storm time (Dst) geomagnetic index (Gruet et al, 2018), and the geomagnetic index Kp (Tan et al, 2018). Different from these pioneer studies, our work focuses on applying the deep learning method of image completion on filling in missing data in space observations.…”

Section: Introductionmentioning

confidence: 99%

Improvement of a Deep Learning Algorithm for Total Electron Content Maps: Image Completion

Zhou

Deng

et al. 2019

JGR Space Physics

View full text Add to dashboard Cite

Observations with a global coverage are very important for space physics research and space weather monitoring. However, due to the technical limitations, it would be very expensive or even impossible to achieve a seamless global coverage even with advanced observational devices. It would be useful to fill missing data gaps to create a global map from the available data, but up until now this has been very challenging. Fortunately, the deep learning method, a recent breakthrough in artificial intelligence, may provide an effective way to solve this problem by making full use of data from reliable observations. In this paper, a promising deep learning algorithm, deep convolutional generative adversarial network (DCGAN), is investigated to fill the missing data of total electron content (TEC) map images. The direct use of DCGAN fails to fill missing data for the completion of TEC maps because there are always missing TEC data in some regions, such as oceans, where the features vary with time and geophysical conditions. Thus, no useful information can be utilized by DCGAN to achieve a meaningful image completion. In order to overcome this shortcoming of the original DCGAN method, a novel regularized DCGAN (R‐DCGAN) is proposed by adding an extra discriminator and some widely used reference TEC maps from the International Global Navigation Satellite Systems Service Ionosphere Working Group. The proposed R‐DCGAN method generates satisfactory ionospheric peak structures at different times and geomagnetic conditions, which demonstrate its effectiveness on filling the missing data in TEC maps. The proposed R‐DCGAN framework can be readily extended to a broad application in other fields of space sciences, particularly for addressing the missing observation data issues.

show abstract

Geomagnetic Index Kp Forecasting With LSTM

Cited by 59 publications

References 14 publications

Nowcasting and Predicting the Kp Index Using Historical Values and Real‐Time Observations

Nowcasting and Predicting the Kp Index Using Historical Values and Real‐Time Observations

The Challenge of Machine Learning in Space Weather: Nowcasting and Forecasting

Improvement of a Deep Learning Algorithm for Total Electron Content Maps: Image Completion

Contact Info

Product

Resources

About