The scope of the Kalman filter for spatio‐temporal applications in environmental science

Abstract

The Kalman filter is a workhorse of dynamical modeling. But there are challenges when using the Kalman filter in environmental science: the complexity of environmental processes, the complicated and irregular nature of many environmental datasets, and the scale of environmental datasets, which may comprise many thousands of observations per time-step. We show how these challenges can be met within the Kalman filter, identifying some situations which are relatively easy to handle, such as datasets which are high-resolution in time, and some which are hard, like areal observations on small contiguous polygons. Overall, we conclude that many applications in environmental science are within the scope of the Kalman filter, or its generalizations.

Environmental data science: Part 1

“…Sparsity is also central to the work of Jurek and Katzfuss (2023), which allows for fast approximate inference with spatio‐temporal models. Computational issues relating to inference with spatio‐temporal models are also considered at length by Rougier et al (2023), who discuss under what conditions sequential updating within each time step is possible, while taking into account the types of data commonly encountered in EDS. Finally, Abdulah et al (2023) show how one can do exact inference with very large datasets; their approach leverages high‐performance computing architectures and parallel linear algebraic libraries, and solves inferential problems that were thought to be practically unsolvable just a few years ago.…”

“…Temporal, spatial and spatio‐temporal models are central to the vast majority of contributions to the issue. Lowther et al (2023) consider multiple time series data that contain change points, and showcase their methods on data on the Greenland ice sheet; Kleiber et al (2023) consider the problem of modeling and simulating tropical cyclone precipitation fields using a spatio‐temporal model in polar coordinates; Shirota et al (2023) tackle the problem of fitting spatial models to light detection and ranging (LiDAR) data collected over Alaska; Abdulah et al (2023) consider the spatial analysis of sea‐surface temperature data; Jurek and Katzfuss (2023) the spatio‐temporal analysis of total precipitable water; Daw and Wikle (2023) the spatial analysis of satellite temperature data; Ning et al (2023) the spatial analysis of presence‐absence ecological data; and the discussion by Rougier et al (2023) focuses on the challenges of fitting spatio‐temporal models to environmental data. The large number of contributed papers involving these classes of models is not coincidental, as many of the phenomena that are analyzed in EDS are temporal, spatial or spatio‐temporal in nature.…”

“…Low-rank representations of process models are often used to address the problem of high dimensionality, and this is the approach taken by Kleiber et al (2023) in the context of circular processes, and by Ning et al (2023), who use resolution-adaptive basis functions for modeling both the covariates and the spatial process of interest. Basis-function expansions also feature in the discussion of Rougier et al (2023). Often, the use of basis functions leads to a problem known as 'over-smoothing'.…”

Scanner: Simultaneously temporal trend and spatial cluster detection for spatial‐temporal data

“…Spatial‐temporal data analysis has attracted increasing research interests and applications in many scientific and engineering fields, such as disease spread analysis (Feng, 2022; Lawson, 2018; Torabi & Rosychuk, 2011; Ugarte et al, 2010), climate data analysis (Erhardt et al, 2015; Velarde et al, 2004; Wan et al, 2021; Zhang et al, 2016), house market study (Gong & de Haan, 2018; Wang et al, 2022), and environmental science (Fioravanti et al, 2022; Johnson et al, 2023; Jurek & Katzfuss, 2023; Rougier et al, 2023; Zhang et al, 2023). In these practical applications, it is an essential step to investigate the underlying spatial‐temporal pattern.…”

Correcting Physics-Based Global Tide and Storm Water Level Forecasts with the Temporal Fusion Transformer

“…Typically, for spatially and/or temporally invariant applications, algorithms like the Extended Kalman filter (Holland, 2020) and Particle filter (Ristic et al, 2003;Li et al, 2017;Elfring et al, 2021) have been leveraged to render these improvements. While they have been adopted for spatiotemporal domains (Butler et al, 2012(Butler et al, , 2015Rougier et al, 2023) and will invariably be scaled for operations in the future, due to computational complexity and expense, they cannot be deployed today for rapid hydrodynamic model improvement. We propose to carry out model improvement by means of machine learning with a transformer model.…”