Characterizing and Visualizing Predictive Uncertainty in Numerical Ensembles Through Bayesian Model Averaging

Abstract

Numerical ensemble forecasting is a powerful tool that drives many risk analysis efforts and decision making tasks. These ensembles are composed of individual simulations that each uniquely model a possible outcome for a common event of interest: e.g., the direction and force of a hurricane, or the path of travel and mortality rate of a pandemic. This paper presents a new visual strategy to help quantify and characterize a numerical ensemble's predictive uncertainty: i.e., the ability for ensemble constituents to accurately and consistently predict an event of interest based on ground truth observations. Our strategy employs a Bayesian framework to first construct a statistical aggregate from the ensemble. We extend the information obtained from the aggregate with a visualization strategy that characterizes predictive uncertainty at two levels: at a global level, which assesses the ensemble as a whole, as well as a local level, which examines each of the ensemble's constituents. Through this approach, modelers are able to better assess the predictive strengths and weaknesses of the ensemble as a whole, as well as individual models. We apply our method to two datasets to demonstrate its broad applicability.

A visual voting framework for weather forecast calibration

“…It is different from the ROI definitions in some previous research, such as regions with high data uncertainty [26] and those with predictive error [6]. More specifically, our ROI definition is to detect the region where errors might exist in the generated initial probabilistic forecast.…”

Comparative visualization of vector field ensembles based on longest common subsequence

“…It can reveal subtle differences between two datasets with different grid resolutions through intermediate mesh. If observation data is available, it can estimate the predictive uncertainty [6], which can help identify potential outlier runs. Clustering is one of most powerful techniques used to extract the uncertainty or other ensemble behavior [20,3].…”

“…Figure 3 depicts a 2D illustration of block-index encoding for two pathlines. The longest common subsequence for these two runs is (0, 1, 2, 2, 3, 6,8,9). We consider all timesteps of the pathline as the elements of the sequence.…”

Uncertainty-Aware Multidimensional Ensemble Data Visualization and Exploration

“…Thomas et al [34] presented an interactive system to study off-shore structures in an ensemble ocean forecasting dataset. Gosink et al [11] proposed a method to characterize different types of predictive uncertainty in ensemble datasets based on the Bayesian model averaging. Mathias et al [12] developed a Lagrangian framework for ensemble flow field analysis.…”

“…However, most of them are only designed for 1D or 2D datasets and have limited capabilities to reveal the intrinsic structures in the ensemble dataset. Recent methods can effectively characterize and analyze the uncertainty structures [9], [10] and forms [11], [12] but are designed for data objects with one variable.…”