Compressing an Ensemble With Statistical Models: An Algorithm for Global 3D Spatio-Temporal Temperature

Abstract

One of the main challenges when working with modern climate model ensembles is the increasingly larger size of the data produced, and the consequent difficulty in storing large amounts of spatio-temporally resolved information. Many compression algorithms can be used to mitigate this problem, but since they are designed to compress generic scientific data sets, they do not account for the nature of climate model output and they compress only individual simulations. In this work, we propose a different, statistics-based approach that explicitly accounts for the space-time dependence of the data for annual global three-dimensional temperature fields in an initial condition ensemble. The set of estimated parameters is small (compared to the data size) and can be regarded as a summary of the essential structure of the ensemble output; therefore, it can be used to instantaneously reproduce the temperature fields in an ensemble with a substantial saving in storage and time. The statistical model exploits the gridded geometry of the data and parallelization across processors. It is therefore computationally convenient and allows to fit a non-trivial model to a data set of one billion data points with a covariance matrix comprising of 10 18 entries.

Reducing storage of global wind ensembles with stochastic generators

“…Statistical models can provide appropriate stochastic approximations of the spatio-temporal characteristics of the model output, and hence they can be used as surrogates of the original runs (Mearns et al, 2001). Castruccio and Stein (2013), Castruccio and Genton (2014), Castruccio and Genton (2016), and Castruccio and Guinness (2017) introduced a Stochastic Generator (SG) for annual temperature data to investigate internal variability for different ensembles, assuming that the observed ensemble members were realizations of an underlying statistical model. This approach allowed them to generate runs that were visually indistinguishable from the original model output.…”

Modeling Temporally Evolving and Spatially Globally Dependent Data

“…Castruccio and Stein () showed that such structure can be used to perform inference on massive data sets from computer model ensembles (in the range of 10 7 to 10 9 data points) by first estimating the spectrum for each of the N ϕ latitudinal bands in parallel, and then conditionally estimating the structure of the spectral correlation ρ ( k ; ϕ , ϕ ′ ). This approach has been extended to analyse three‐dimensional temperature profiles in a regular grid for a data set larger than one billion data points, allowing for an extension of axially symmetric models in three dimensions (Castruccio & Genton, ).…”

Visualizing Spatiotemporal Models with Virtual Reality: from Fully Immersive Environments to Applications in Stereoscopic View

“…The headset is equipped with a wireless motion capture device to allow each user to explore the data within the environment interactively. In a recent workshop, a model for three‐dimensional temperature was fitted and simulations from a statistical model (or emulator ; full model details are given in Castruccio and Genton ()) were compared with the original climate model. An interactive display was shown.…”