Removing Diurnal Signals and Longer Term Trends From Electron Flux and ULF Correlations: A Comparison of Spectral Subtraction, Simple Differencing, and ARIMAX Models

Abstract

Periodicity is common in space weather data, with cycles ranging from diurnal (24 hr), the 27 days solar rotation, and to the 11 yr solar cycle. Parameters will be highly correlated if time series variables cycle in parallel, or if both follow a long-term trend, but this correlation may say nothing about the more immediate, causal relationship between them. As time series can be thought of as compositions of trends and cycles, it is possible to remove these by subtraction once they are identified (Hyndman & Athanasopoulos, 2018).Both geosynchronous and ground-based measurements may depend on magnetic local time and thus exhibit diurnal variations. Although high energy electron fluxes remain mostly stable along any given drift shell, satellites at geosynchronous orbit, due to the asymmetric dipole of the Earth's magnetic field, do not stay within the same drift

Using ARMAX Models to Determine the Drivers of 40–150 keV GOES Electron Fluxes

“…However, it may be that the flux response to each predictor varies depending on conditions. A simpler multiple regression model could be used to explore the response between quiet and disturbed periods, however, this can result in spurious correlations if variables are cycling together (e.g., a diurnal cycle) or show a common trend (Simms et al., 2022). A regression model that accounts for these co‐occurring cycles and trends can be produced by differencing the data: subtracting the previous value from each observation ( y t − y t −1 ).…”

“…In the undifferenced data, we do find the “expected” strong ULF effect (Figure 6.2; note the larger scale compared to the differenced data), but this is only a demonstration of the spurious nature of this high correlation. High correlations between ULF wave activity and electron flux in hourly data are likely only describing a common diurnal cycle and say little about physical driving mechanisms (Simms et al., 2022). Note that it is not so much that the correlation is “wrong” but that the differencing or ARMA modeling removes the portion of the correlation that is irrelevant to the questions we are interested in.…”

“…In the present study, we test the influence of several possible drivers of low energy electron flux (40–150 keV) observed by GOES 13 and GOES 16 satellites: solar wind velocity ( V ), number density ( N ), pressure ( P ), and the electric field ( E y ), IMF B z , and substorms (as measured by the AE index). We use ARMAX (autoregressive moving average transfer function) models both to measure the cumulative effects and to remove common cycles and trends that may inflate correlations between variables (Simms et al., 2022). These parameters may act in combination, with influence accumulating over time.…”

“…MeV (mega electronvolt) electron fluxes at GEO have been more extensively studied and may show high overall correlations with solar wind parameters when daily averaged (e.g., Blake et al, 1997), although the hourly response may be much lower (Simms et al, 2022). Solar wind speed is often cited as the most important driver (Li et al, 2001;Paulikas & Blake, 1979), although the relationship is complex (Reeves et al, 2011) and, for example, Lyatsky and Khazanov (2008) and Balikhin et al (2011) have shown that the solar wind density is more associated with MeV electron variations.…”

“…Other studies have used ARMAX modeling to predict higher energy electron fluxes in geostationary orbit, and these provide further information on this approach of describing the underlying cyclical behavior of flux with AR and MA terms (Balikhin et al, 2011(Balikhin et al, , 2016Simms & Engebretson, 2020;Simms et al, 2022). We note that nonlinearity was introduced into the models of Balikhin et al (2011) with polynomial terms (square and cubic terms) instead of the logs we use here.…”