CAiTST: Conv-Attentional Image Time Sequence Transformer for Ionospheric TEC Maps Forecast

Abstract

In recent years, transformer has been widely used in natural language processing (NLP) and computer vision (CV). Comparatively, forecasting image time sequences using transformer has received less attention. In this paper, we propose the conv-attentional image time sequence transformer (CAiTST), a transformer-based image time sequences prediction model equipped with convolutional networks and an attentional mechanism. Specifically, we employ CAiTST to forecast the International GNSS Service (IGS) global total electron content (TEC) maps. The IGS TEC maps from 2005 to 2017 (except 2014) are divided into the training dataset (90% of total) and validation dataset (10% of total), and TEC maps in 2014 (high solar activity year) and 2018 (low solar activity year) are used to test the performance of CAiTST. The input of CAiTST is presented as one day’s 12 TEC maps (time resolution is 2 h), and the output is the next day’s 12 TEC maps. We compare the results of CAiTST with those of the 1-day Center for Orbit Determination in Europe (CODE) prediction model. The root mean square errors (RMSEs) from CAiTST with respect to the IGS TEC maps are 4.29 and 1.41 TECU in 2014 and 2018, respectively, while the RMSEs of the 1-day CODE prediction model are 4.71 and 1.57 TECU. The results illustrate CAiTST performs better than the 1-day CODE prediction model both in high and low solar activity years. The CAiTST model has less accuracy in the equatorial ionization anomaly (EIA) region but can roughly predict the features and locations of EIA. Additionally, due to the input only including past TEC maps, CAiTST performs poorly during magnetic storms. Our study shows that the transformer model and its unique attention mechanism are very suitable for images of a time sequence forecast, such as the prediction of ionospheric TEC map sequences.

Recent Advances in Modelling Geodetic Time Series and Applications for Earth Science and Environmental Monitoring

“…that are contaminated by various types of noise (i.e., stochastic processes and correlations). Several papers have contributed to GNSS and its application to crustal deformation and geodynamics [63][64][65][66][67]; civil engineering [68,69]; stochastic noise modelling [70,71]; natural hazards such as landslides [36,37,72]; SLR estimation and coastal flooding [73][74][75]; hydrology, seasonal displacements, and drought monitoring using GNSS and/or GRACE/GRACE-FO [76][77][78]; and the study of ionospheric disturbances [79][80][81], together with research focused on the stability of the reference frame [82].…”

Forecasting of Global Ionosphere Maps With Multi‐Day Lead Time Using Transformer‐Based Neural Networks

“…(2022), Xia, Liu, et al. (2022), and Xia, Zhang, et al. (2022) proposed a novel model to predict the time series of global TEC maps using an encoder‐decoder framework based on ConvLSTM network.…”

“…Finally, Xia, Liu, et al. (2022) and Xia, Zhang, et al. (2022) proposed the conv‐attentional image time sequence Transformer, a Transformer‐based (Vaswani et al., 2017) image time sequences prediction model equipped with convolutional networks and an attention mechanism.…”

Forecast Global Ionospheric TEC: Apply Modified U‐Net on VISTA TEC Data Set

“…These approaches are promising in solving nonlinear prediction problems (Han et al., 2021) and therefore can predict TEC more accurately (e.g., L. Liu et al., 2020). Various models were developed for single‐station (e.g., Huang & Yuan, 2014; Huang et al., 2015; Tebabal et al., 2018), regional (Ferreira et al., 2017; Song et al., 2018; Tebabal et al., 2019; Xia et al., 2021), and global TEC forecast (Cesaroni et al., 2020; Chen et al., 2022; Lee et al., 2021; L. Liu et al., 2020, 2022; J. Tang et al., 2022; Xia, Liu, et al., 2022; Xia, Zhang, et al., 2022; Yang et al., 2022). Unlike single‐station and regional forecasts that are usually trained with time series of TEC from different stations, global forecast models need to be trained with global TEC maps.…”