Process-Oriented Evaluation of Climate and Weather Forecasting Models

Maloney, Eric D.; Gettelman, Andrew; Ming, Yi; Neelin, J. David; Barrie, Daniel; Mariotti, Annarita; Chen, Celia; Coleman, D.; Kuo, Yao Haur; Singh, Bohar; Annamalai, H.; Berg, Alexis; Booth, James F.; Camargo, Suzana J.; Dai, Aiguo; Gonzalez, Alejandro; Hafner, Jan; Jiang, Xianan; Jing, Xianwen; Kim, Dae Hyun; Kumar, Arun; Moon, Yumin; Naud, Catherine M.; Sobel, Adam H.; Suzuki, Kentaroh; Wang, Fuchang; Wang, Junhong; Wing, Allison A.; Xu, Xiaobiao; Zhao, Ming

doi:10.1175/bams-d-18-0042.1

Cited by 56 publications

(52 citation statements)

References 96 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Better understanding the sources of errors in MJO processes is the key to improving MJO prediction. Process‐based diagnostics aim to ascertain which physical processes should be targeted for improvement in models to better simulate the MJO and help focus model development efforts (e.g., Maloney et al, ). However, process‐based model evaluations of MJO prediction are limited in number and largely limited to case studies of individual MJO events (Hannah et al, ; Hannah & Maloney, ; Klingaman et al, ; Ling et al, ).…”

Section: Introductionmentioning

confidence: 99%

MJO Propagation Processes and Mean Biases in the SubX and S2S Reforecasts

Kim

Janiga²,

Pegion

2019

JGR Atmospheres

View full text Add to dashboard Cite

The Madden‐Julian oscillation (MJO) is the leading source of global subseasonal predictability; however, many dynamical forecasting systems struggle to predict MJO propagation through the Maritime Continent. Better understanding the biases in simulated physical processes associated with MJO propagation is the key to improve MJO prediction. In this study, MJO prediction skill, propagation processes, and mean state biases are evaluated in reforecasts from models participating in the Subseasonal Experiment (SubX) and Subseasonal to Seasonal (S2S) prediction projects. SubX and S2S reforecasts show MJO prediction skill out to 4.5 weeks based on the Real‐time Multivariate MJO index consistent with previous studies. However, a closer examination of these models' representation of MJO propagation through the Maritime Continent reveals that they fail to predict the MJO convection, associated circulations, and moisture advection processes beyond 10 days with most of models underestimating MJO amplitude. The biases in the MJO propagation can be partly associated with the following mean biases across the Indo‐Pacific: a drier low troposphere, excess surface precipitation, more frequent occurrence of light precipitation rates, and a transition to stronger precipitation rates at lower humidity than in observations. This indicates that deep convection occurs too frequently in models and is not sufficiently inhibited when tropospheric moisture is low, which is likely due to the representation of entrainment.

show abstract

Section: Introductionmentioning

confidence: 99%

MJO Propagation Processes and Mean Biases in the SubX and S2S Reforecasts

Kim

Janiga²,

Pegion

2019

JGR Atmospheres

View full text Add to dashboard Cite

show abstract

“…This study was supported by the Modeling, Analysis, Predictions, and Projections (MAPP) program of the NOAA Climate Program Office (CPO) through Grants NA15OAR4310087, NA15OAR4310095, NA18OAR4310270, NA18OAR4310276, and NA18OAR4310277. This work is a contribution to the process-oriented diagnostics efforts of the NOAA MAPP Model Diagnostics Task Force (e.g., Maloney et al 2019).…”

mentioning

confidence: 99%

Azimuthally Averaged Wind and Thermodynamic Structures of Tropical Cyclones in Global Climate Models and Their Sensitivity to Horizontal Resolution

et al. 2020

Self Cite

View full text Add to dashboard Cite

Characteristics of tropical cyclones (TCs) in global climate models (GCMs) are known to be influenced by details of the model configurations, including horizontal resolution and parameterization schemes. Understanding model-to-model differences in TC characteristics is a prerequisite for reducing uncertainty in future TC activity projections by GCMs. This study performs a process-level examination of TC structures in eight GCM simulations that span a range of horizontal resolutions from 1° to 0.25°. A recently developed set of process-oriented diagnostics is used to examine the azimuthally averaged wind and thermodynamic structures of the GCM-simulated TCs. Results indicate that the inner-core wind structures of simulated TCs are more strongly constrained by the horizontal resolutions of the models than are the thermodynamic structures of those TCs. As expected, the structures of TC circulations become more realistic with smaller horizontal grid spacing, such that the radii of maximum wind (RMW) become smaller, and the maximum vertical velocities occur off the center. However, the RMWs are still too large, especially at higher intensities, and there are rising motions occurring at the storm centers, inconsistently with observations. The distributions of precipitation, moisture, and radiative and surface turbulent heat fluxes around TCs are diverse, even across models with similar horizontal resolutions. At the same horizontal resolution, models that produce greater rainfall in the inner-core regions tend to simulate stronger TCs. When TCs are weak, the radial gradient of net column radiative flux convergence is comparable to that of surface turbulent heat fluxes, emphasizing the importance of cloud–radiative feedbacks during the early developmental phases of TCs.

show abstract

“…To answer this, we hope to develop some physical-process oriented tools within MATS, which allow the model-observation pairs to be saved with various discriminators that allow the data to be interrogated in different ways (e.g., as a function of stability, wind direction, etc.). Processoriented verification is becoming more popular in both the weather and climate communities (e.g., Maloney et al 2019). The challenge here is one of data volume and subsequent system responsivity; some partial sums need to be computed to reduce the dimensionality of the data, but the data should not be reduced so much that we lose the ability to generate statistics for various scenarios determined by the model's use of the discriminators.…”

Section: Futurementioning

confidence: 99%

A Verification Approach Used in Developing the Rapid Refresh and Other Numerical Weather Prediction Models

Turner

Hamilton

Moninger

et al. 2020

J. Operational Meteor.

View full text Add to dashboard Cite

Developing and improving numerical weather prediction models such as the Rapid Refresh (RAP) and High-Resolution Rapid Refresh (HRRR) requires a well-designed, easy-to-use evaluation capability using observations. Owing to the very complex nonlinear interactions between the data assimilation system and the representation of various physics components in the model, changes to one aspect of the modeling system to address a particular shortcoming within the model may have detrimental impacts in another area. Thus, the model verification approach used in the Global Systems Division of the NOAA Earth System Research Laboratory—which actively develops the RAP and HRRR models and other forecasting systems—is designed to allow hypothesis-driven testing of different aspects of the model using observations. In this approach, model changes easily and quickly can be quantified by automatically comparing simulated geophysical variables against many different types of observations that are collected operationally by various agencies, including the National Weather Service. We have implemented this approach in the Model Analysis Tool Suite (MATS). A key aspect of MATS is the use of a database-driven system that stores partial sums of model minus observation pairs over specified geographical regions in order to reduce the dimensionality of the data and, thus, improve the response time of the system. These partial sums are created and stored in a manner that allows the data to be visualized in different ways, thereby providing new insights into the ability of that particular version of the model to replicate the observed atmospheric conditions.

show abstract

Process-Oriented Evaluation of Climate and Weather Forecasting Models

Cited by 56 publications

References 96 publications

MJO Propagation Processes and Mean Biases in the SubX and S2S Reforecasts

MJO Propagation Processes and Mean Biases in the SubX and S2S Reforecasts

Azimuthally Averaged Wind and Thermodynamic Structures of Tropical Cyclones in Global Climate Models and Their Sensitivity to Horizontal Resolution

A Verification Approach Used in Developing the Rapid Refresh and Other Numerical Weather Prediction Models

Contact Info

Product

Resources

About