Spatiotemporal Attention for Multivariate Time Series Prediction and Interpretation

Lin

Int J of Intelligent Sys

et al. 2022

Time series research in academic and industrial fields has attracted wide attention. However, the frequency information contained in time series still lacks effective modeling. The studies found that time series forecasting relies on different frequency patterns: short-term series forecasting relies more on high-frequency components, while long-term forecasting focuses more on low-frequency data. To better describe the multifrequency mode, a dual-stage multifeature adaptive frequency domain prediction model (DMAFD) is proposed in this paper. DMAFD contains two stages. First, it adopts the XGBoost algorithm to obtain a feature vector by analyzing the feature importance. Second, the frequency feature extraction of time series and the frequency aware modeling of the target sequence is integrated, for building an end-to-end prediction network based on the dependence of time series on frequency mode. The innovation is reflected in the fact that the prediction network can automatically focus on multifrequency components according to the dynamic evolution of the input sequence. Extensive experiments on four real data sets from different fields show that DMAFD obtains higher accuracy and smaller lags in time step analysis compared with state-of-the-art algorithms.

Section: Baseline Comparisonmentioning

confidence: 99%

Section: Experiments and Analysismentioning

confidence: 99%

Dual‐stage time series analysis on multifeature adaptive frequency domain modeling

Lin

Int J of Intelligent Sys

et al. 2022

“…Compressing all information from the input time-steps into a fixed-length single vector was the major bottleneck for the encoder-decoder model. Temporal attention can be applied for many-to-many time series prediction 24 and many-to-one-prediction 39,40 . The proposed approach (Fig.…”

Section: Model Developmentmentioning

confidence: 99%

“…LSTMs have shown state-of-the-art results in various applications including off-line handwriting recognition 20 , natural language processing 21 and engineering systems 22 . LSTMs have also been used effectively for multivariate time series prediction tasks [23][24][25] . Considering the importance of climate extremes for agricultural predictions, random forest has been utilized to predict grid-cell anomalies-deviations of yields 26 .…”

Section: Introductionmentioning

confidence: 99%

Crop Yield Prediction Integrating Genotype and Weather Variables Using Deep Learning

Shook,

Gangopadhyay,

et al. 2020

Preprint

Self Cite

Accurate prediction of crop yield supported by scientific and domain-relevant insights, can help improve agricultural breeding, provide monitoring across diverse climatic conditions and thereby protect against climatic challenges to crop production including erratic rainfall and temperature variations. We used historical performance records from Uniform Soybean Tests (UST) in North America spanning 13 years of data to build a Long Short Term Memory -Recurrent Neural Network based model to dissect and predict genotype response in multiple-environments by leveraging pedigree relatedness measures along with weekly weather parameters. Additionally, for providing explainability of the important time-windows in the growing season, we developed a model based on temporal attention mechanism. The combination of these two models outperformed random forest (RF), LASSO regression and the data-driven USDA model for yield prediction. We deployed this deep learning framework as a 'hypotheses generation tool' to unravel GxExM relationships. Attention-based time series models provide a significant advancement in interpretability of yield prediction models. The insights provided by explainable models are applicable in understanding how plant breeding programs can adapt their approaches for global climate change, for example identification of superior varieties for commercial release, intelligent sampling of testing environments in variety development, and integrating weather parameters for a targeted breeding approach. Using DL models as hypothesis generation tools will enable development of varieties with plasticity response in variable climatic conditions. We envision broad applicability of this approach (via conducting sensitivity analysis and "what-if" scenarios) for soybean and other crop species under different climatic conditions.

“…Recently, machine learning (ML), has become one of the most powerful tools in the field of multivariate multi-step time series prediction (Hochreiter and Schmidhuber, 1997;Geurts et al, 2006;Sapankevych and Sankar, 2009;Box et al, 2015;Hu and Zheng, 2020;Nian et al, 2021c). Deep learning could be regarded as one of the hottest topics in the context, and all kinds of most emerging and advanced algorithms have been put forward and made progresses (Hinton and Salakhutdinov, 2006;Krizhevsky et al, 2012;Goodfellow et al, 2014;He et al, 2016;Huang et al, 2017;Wan et al, 2019), such as Recurrent Neural Network (RNN) (Elman, 1990;Lipton et al, 2015;Braakmann-Folgmann et al, 2017;Qin et al, 2017) and Long Short Term Memory (LSTM) (Kalchbrenner et al, 2015;Shi et al, 2015;Greff et al, 2016;Zhang et al, 2017;Shi and Yeung, 2018;Wang et al, 2021;Gangopadhyay et al, 2021;Nian et al, 2021b). We expect to establish a comprehensive predictive model of mesoscale eddy trajectories toward meridional ridges on a global scale in the future, coupling with the topographic effects, via deep learning.…”

Section: Introductionmentioning

confidence: 99%

Toward Exploring Topographic Effects on Evolution and Propagation of Ocean Mesoscale Eddies Through Life Cycle Across Izu-Ogasawara Ridge in Northwestern Pacific Ocean

et al. 2022

Ocean mesoscale eddies contribute significantly to water transport on a global scale, constituting the ubiquitous, irregular, discrete, nonlinear components. In this manuscript, we propose to explore whether and how the topographic effect of one meridional ridge, could exert considerable influences on the evolution and propagation of mesoscale eddies through their life cycle, directly from the perspectives of real observation statistics. We systematically investigate the known variability of mesoscale eddy trajectories, derived by multimission satellite altimetry from 1993 to 2018, of a life cycle more than 6 months, over the Izu-Ogasawara Ridge, and quantitatively examined the eddy-ridge interaction by observation statistics and wavelet coherence map, with respect to the intrinsic attributes, namely, the amplitude, the rotation speed, the radius. Due to the spatial-temporal diversity, a series of correlative steps have been particularly designed along time-frequency domain to trace back mesoscale eddy trajectories in a variety of origins, location, lifespan, polarity, either completely or partially passing over the ridge, and to facilitate the standardization in statistics across three phases of their life cycle, i.e., before, during and after the interaction with the ridge. It has been revealed in our experiment that three intrinsic attributes of mesoscale eddies within 25 years, all demonstrated noticeable correlation with the variation of topographic relief over the ridge. We observed that most of the cyclonic eddies obviously tended to begin to decay or even demise, while on the contrary, some of the anticyclonic eddies preferred to intensify slightly, or making no significant difference when encountering the upslope until climbing across the top, basically consistent with the expectation of potential vorticity (PV) conservation. The drifting velocity agreed with the tendency that the direction would be more probably modified toward equatorward or poleward by forcing to meridional component, with zonal component reduced at the beginning. The mesoscale eddies with the passage over the ridge exhibited the relatively high average horizontal scales, amplitude, rotation speed on the whole, compared to those with only partially passage. The developed scheme could integrate more evidences on how mesoscale eddies response to the topographic effects during their time-varying evolution and propagation process, and help provide opportunities to potentially identify and predict the underlying dynamic patterns and mechanisms that mesoscale eddies engage in ocean dynamics when proceeding toward meridional ridges on a global scale, with the promise of the end-to-end data-driven solution, such as deep learning architecture involved in the future.