A Data Quality Control Method Using Forecasted Horizontal Gradient and Tendency in a NWP System

Abstract

A method of quality control (QC) to reject erroneous observational data has been developed in the Numerical Weather Prediction System of the Japan Meteorological Agency (JMA). Each observed value is processed in the QC, by evaluating its deviation from the first guess (FG) value given by the forecasted field, comparing it with a threshold value defined by statistics and rejecting it if it falls outside the threshold range. In the previous method (Static QC), the threshold value was defined as a constant regardless of the weather conditions.In the new method (Dynamic QC, in this paper), the threshold value is defined as a variable linearly depending on the local horizontal gradient and three-hour tendency of the FG. With a statistical study, we found a relationship between the deviations, and both the horizontal gradient and the tendency of FG were approximately proportional. Then this relationship was applied to the Dynamic QC. In almost all regions, the Dynamic QC reduces erroneous judgements given in the Static QC where wrong observed values were accepted in less disturbed weather conditions and right values were rejected in more disturbed conditions. Forecast experiments using the Dynamic QC revealed that the forecast scores and the fit in root mean square error (RMSE) to rawinsonde observations were improved in the tropics and the Southern Hemisphere in winter. For some cases, the improvement of the scores in Anomaly Correlation Coefficient (ACC) was equivalent to more than one days extension of the forecast term, and the ratios of fit in RMSE were improved by more than 10 %. The forecast results for the Northern Hemisphere were neutral. The Dynamic QC method was put into operations at JMA on March 17, 1997. After that, the scores of the operational global forecasts have significantly improved.

Accounting for Flow-dependence in the Background Error Variance within the JMA Global Four-dimensional Variational Data Assimilation System

“…The reliability is measured by the horizontal gradient or temporal tendency of the background field, which varies in time and space. Onogi (1998) confirmed that the horizontal gradient and temporal tendency were correlated well with the observation departure; namely, they compose good measures of reliability in fact. Figure 2 (a) shows a horizontal map of vorticity error variance weighted by the horizontal gradient of the vorticity field on 00 UTC, 18 August 2006.…”

The JRA-55 Reanalysis: General Specifications and Basic Characteristics

“…A notable feature of the QC methods used in JMA's data assimilation system is "Dynamic QC," in which the threshold value is defined as a variable linearly depending on the local horizontal gradient and tendency of the first-guess fields (Onogi 1998). Since JRA-25, the set of QC thresholds has been thoroughly reviewed and updated (Sakamoto 2009).…”

The JRA-25 Reanalysis

“…These QC systems also try to automatically correct erroneous data when possible. JMA originally developed ''Dynamic QC'' techniques (Onogi 1998) based on departure statistics for most types of data. The separate ''Group QC'' (Nomura and Tahara 1997) is also developed and applied for sea surface winds from scatterometer data to fix the problem of wind direction ambiguity.…”