Statistical Design for Adaptive Weather Observations

Abstract

Suppose that one has the freedom to adapt the observational network by choosing the times and locations of observations. Which choices would yield the best analysis of the atmospheric state or the best subsequent forecast? Here, this problem of ''adaptive observations'' is formulated as a problem in statistical design. The statistical framework provides a rigorous mathematical statement of the adaptive observations problem and indicates where the uncertainty of the current analysis, the dynamics of error evolution, the form and errors of observations, and data assimilation each enter the calculation. The statistical formulation of the problem also makes clear the importance of the optimality criteria (for instance, one might choose to minimize the total error variance in a given forecast) and identifies approximations that make calculation of optimal solutions feasible in principle. Optimal solutions are discussed and interpreted for a variety of cases. Selected approaches to the adaptive observations problem found in the literature are reviewed and interpreted from the optimal statistical design viewpoint. In addition, a numerical example, using the 40-variable model of Lorenz and Emanuel, suggests that some other proposed approaches may often be close to the optimal solution, at least in this highly idealized model.

An approach to designing a national climate service

“…At the national scale, the NCS would assist in coordinating the upgrade, expansion, and/or optimization of existing and future observation networks. This point is key: new modeling and analytical techniques permit very sophisticated methods of determining the optimal placement and instrumentation for multiple objectives for a given cost and may permit the retirement of some existing observations and the establishment of much-needed new observations (14,15).…”

Data assimilation in atmospheric chemistry models: current status and future prospects for coupled chemistry meteorology models

“…Nevertheless, the dynamic placement of new and informative observations (targeting) has been investigated theoretically (Berliner et al, 1999; and many since then) and experimentally in field campaigns such as the Fronts and Atlantic Storm-Track Experiment (FASTEX) of Meteo France (http://www.cnrm.meteo.fr/dbfastex/ftxinfo/) and the Observing System Research and Predictability Experiment of the World Meteorological Organization (THORPEX; http://www.wmo.int/pages/prog/arep/wwrp/ new/THORPEXProjectsActivities.html). Although these adaptive observations were shown to be very informative in the case of severe events, they are based on monitoring flights and hence are very costly, whereas other observations are much more abundant and cheaper.…”

Learning the Covariance Dynamics of a Large-Scale Environment for Informative Path Planning of Unmanned Aerial Vehicle Sensors

“…Furthermore, measurements at different locations and times possess different amounts of information due to the spatiotemporal correlations of the state variables. To make the most effective use of finite resources, we must consider the value of the possible measurements in terms of prediction performance (Berliner et al, 1999;Choi and How, 2010a, b;Morss et al, 2001), and choose to measure the state variables that maximize the prediction accuracy.…”

“…This can be easily extended to a multiagent problem where each agent chooses a path, and the collective information gain on V by these agents is optimized. Typically, trace or entropy of the covariance matrix are used as the measures of uncertainty (Berliner et al, 1999;Choi and How, 2010a). We use trace in this work, but the work can be generalized to entropy.…”

“…The path selection problem (or "informative path planning" problem) may be straightforwardly solved if the posterior covariance can be computed efficiently subsequent to taking a sequence of the measurements (Berliner et al, 1999;Choi and How, 2010b). When the system dynamics are linear (and any perturbations to the system or to the measurements are Gaussian), the posterior covariance given a sequence of measurements can be easily and accurately computed using the Kalman filter (EKF).…”