Multivariable integrated evaluation of model performance with the vector field evaluation diagram

Xu, Zhongfeng; Han, Ying; Fu, Congbin

doi:10.5194/gmd-10-3805-2017

Cited by 14 publications

(18 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To meet these demands, Xu et al . () developed a multivariable integrated evaluation (MVIE) framework that includes three levels of statistical metrics and can provide a more comprehensive and quantitative evaluation of model performance.…”

Section: Discussionmentioning

confidence: 99%

“…Meanwhile, one may also want to examine the influence of observational uncertainties on the evaluation results. To meet these demands, Xu et al (2017) developed a multivariable integrated evaluation (MVIE) framework that includes three levels of statistical metrics and can provide a more comprehensive and quantitative evaluation of model performance.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Short communication comments on ‘DISO: A rethink of Taylor diagram’

Han

2019

Intl Journal of Climatology

Self Cite

View full text Add to dashboard Cite

In a recent article, Hu et al. (2019) argued that the commonly used Taylor diagram in climate model evaluation only considered centred statistics, while the absolute error (AE) was not considered. Therefore, they proposed a new index, termed Distance between Indices of Simulation and Observation (DISO), that takes AE, correlation coefficient, and uncentred RMSE into account to comprehensively measure climate model performance. Note that the widely used uncentred RMSE is a function of the AE and correlation coefficient. Therefore, it is not necessary to combine AE and correlation coefficient with the uncentred RMSE again to define DISO. In addition, model performance does not improve monotonically with the decrease in the DISO index, which is also a flaw of the index. Indeed, the Taylor diagram was constructed based on the cosine law between three‐centred statistics that cannot measure the mean error of the climate model. However, this limitation can be mitigated by showing the mean error or percent bias on the diagram or constructing the Taylor diagram with uncentred statistics. The uncentred statistics still satisfy the cosine law and can also be illustrated in the Taylor diagram. In our viewpoint, there is nothing wrong with the Taylor diagram. On the contrary, the Taylor diagram can effectively visualize multiple statistics and compare the performance of different climate models.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Short communication comments on ‘DISO: A rethink of Taylor diagram’

Han

2019

Intl Journal of Climatology

Self Cite

View full text Add to dashboard Cite

show abstract

“…Among the satellite-based precipitation products, the TRMM-TMPA 3B42V7 (Huffman et al, 2007) and GPM-IMERG (Hou et al, 2014) are generally recognized as the reliable precipitation data evaluated against in situ observations (Koo et al, 2009;Li et al, 2017;Ma et al, 2016;Maussion et al, 2011Maussion et al, , 2014Sato et al, 2008;Tong et al, 2014;Xu, Han, et al, 2017;Zhang et al, 2018). Specifically, the TRMM-TMPA 3B42V7 data merges gauged monthly rainfall and is the best product for the TRMM-era (Ma et al, 2016).…”

Section: Methodsmentioning

confidence: 99%

Cloud Resolving WRF Simulations of Precipitation and Soil Moisture Over the Central Tibetan Plateau: An Assessment of Various Physics Options

Yang

2020

Earth and Space Science

Self Cite

View full text Add to dashboard Cite

The evaluation of the regional climate model is of great importance for model's developments and applications. We assessed the performance of Weather Research and Forecasting (WRF) cloud resolving simulations with various physics options in terms of precipitation and soil moisture over the central Tibetan Plateau (TP) for a 2‐month simulation from July to August in 2015. The simulated precipitation is most sensitive to the microphysics scheme, followed by the land surface model, which plays a vital role in the soil moisture simulation, while the planetary boundary layer and radiation schemes have relatively minor impacts on the precipitation and soil moisture. Specifically, the heavy precipitation event has a close relationship with the land surface model. Among the different WRF schemes, the new Thompson microphysics scheme, the Noah land surface model, the GFDL radiation scheme, and the Mellor–Yamada planetary boundary layer scheme perform relatively better than other options over the central TP. In contrast, the Lin and WRF Single‐Moment 6‐class microphysics schemes tend to simulate an earlier precipitation peak in the diurnal cycle, excessively higher intensities, and greater frequencies for high precipitation events. The Rapid Update Cycle model performs the worst in the spatiotemporal pattern of precipitation and markedly exaggerates the diurnal variation of soil moisture. These results can provide valuable guidance for further fine‐scale simulation studies of land–atmosphere interaction over the TP.

show abstract

“…The VFE method allows an evaluation of a model's ability to simulate a vector field (Huang et al, 2019(Huang et al, , 2020. Based on the VFE diagram, Xu et al (2017) further developed a multivariable in-M.-Z. Zhang et al: MVIETool version 1.0 tegrated evaluation (MVIE) method to evaluate model performance in terms of multiple fields by grouping various normalized scalar fields into an integrated vector field.…”

Section: Introductionmentioning

confidence: 99%

An improved multivariable integrated evaluation method and tool (MVIETool) v1.0 for multimodel intercomparison

Zhang

Han

et al. 2021

Geosci. Model Dev.

Self Cite

View full text Add to dashboard Cite

Abstract. An evaluation of a model's overall performance in simulating multiple fields is fundamental to model intercomparison and development. A multivariable integrated evaluation (MVIE) method was proposed previously based on a vector field evaluation (VFE) diagram, which can provide quantitative and comprehensive evaluation on multiple fields. In this study, we make further improvements to this method from the following aspects. (1) We take area weighting into account in the definition of statistics in the VFE diagram and MVIE method, which is particularly important for a global evaluation. (2) We consider the combination of multiple scalar fields and vector fields against multiple scalar fields alone in the previous MVIE method. (3) A multivariable integrated skill score (MISS) is proposed as a flexible index to measure a model's ability to simulate multiple fields. Compared with the multivariable integrated evaluation index (MIEI) proposed in the previous study, MISS is a normalized index that can adjust the relative importance of different aspects of model performance. (4) A simple-to-use and straightforward tool, the Multivariable Integrated Evaluation Tool (MVIETool version 1.0), is developed to facilitate an intercomparison of the performance of various models. Users can use the tool coded either with the open-source NCAR Command Language (NCL) or Python3 to calculate the MVIE statistics and plotting. With the support of this tool, one can easily evaluate model performance in terms of each individual variable and/or multiple variables.

show abstract

Multivariable integrated evaluation of model performance with the vector field evaluation diagram

Cited by 14 publications

References 19 publications

Short communication comments on ‘DISO: A rethink of Taylor diagram’

Short communication comments on ‘DISO: A rethink of Taylor diagram’

Cloud Resolving WRF Simulations of Precipitation and Soil Moisture Over the Central Tibetan Plateau: An Assessment of Various Physics Options

An improved multivariable integrated evaluation method and tool (MVIETool) v1.0 for multimodel intercomparison

Contact Info

Product

Resources

About