We address inconsistent procedures and metrics used to evaluate photochemical model performance, recommend a specific set of statistical metrics, and develop updated quantitative performance benchmarks for those metrics. We promote quantitatively consistent evaluations across different applications, scales, models, inputs, and configurations, thereby (1) improving the user's ability to quantitatively place results in context and guide model improvements, and (2) better informing users, regulators, and stakeholders of model uncertainties and weaknesses prior to using results for policy assessments. While we primarily address U.S. modeling and regulatory settings, these recommendations are relevant to any such applications of state-of-the-science photochemical models.
Guidance for the performance evaluation of three-dimensional air quality modeling systems for particulate matter and IMPLICATIONS The National Ambient Air Quality Standards for particulate matter (PM) and the federal regional haze regulations place some emphasis on the assessment of fine particle (PM 2.5 ) concentrations. Current air quality models need to be improved and evaluated against observations to assess the reliability of model simulations. The guidance presented here provides the necessary framework for conducting rigorous performance evaluations of PM and visibility models. The costs associated with the field programs needed to obtain the data necessary for such performance evaluations are estimated to be $15 million for data collection (1-year program with an intensive program of 15 days, over 200,000 km 2 ) and $10-$20 million for planning, emission inventories, data analysis, and modeling. visibility is presented. Four levels are considered: operational, diagnostic, mechanistic, and probabilistic evaluations. First, a comprehensive model evaluation should be conducted in at least two distinct geographical locations and for several meteorological episodes. Next, streamlined evaluations can be conducted for other similar applications if the comprehensive evaluation is deemed satisfactory. In all cases, the operational evaluation alone is insufficient, and some diagnostic evaluation must always be carried out. Recommendations are provided for designing field measurement programs that can provide the data needed for such model performance evaluations.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.