An evaluation of diagnostic tests and their roles in validating forest biometric models

Abstract

Model validation is an important part of model development. It is performed to increase the credibility and gain sufficient confidence about a model. This paper evaluated the usefulness of 10 statistical tests, five parametric and five nonparametric, in validating forest biometric models. The five parametric tests are the paired t test, the Χ2 test, the separate t test, the simultaneous F test, and the novel test. The five nonparametric tests are the Brown-Mood test, the Kolmogorov–Smirnov test, the modified Kolmogorov–Smirnov test, the sign test, and the Wilcoxon signed-rank test. Nine benchmark data sets were selected to evaluate the behavior of these tests in model validation; three were collected from Alberta and six were published elsewhere. It was shown that the usefulness of statistical tests in model validation is very limited. None of the tests seems to be generic enough to work well across a wide range of models and data. Each model passed one or more tests, but not all of them. Because of this, caution should be exercised when choosing a statistical test or several tests together to try to validate a model. It is important to reduce and remove any potential personal bias in selecting a favorite test, which can influence the outcome of the results.

Un modèle de croissance pour des peuplements de Pinus radiata D. Don du nord ouest de l’Espagne

“…Although the use of biological and theoretical criteria is important in model evaluation, the ability of a model to represent adequately the real world is normally addressed through model validation [90]. Ideally, such validation should involve the use of an independent data set [55,63,100,103]. Moreover, variations in stand age and environmental factors must be included in the data set [13,81,94].…”

Estimation de la croissance dans les hêtraies : une étude basée sur des expérimentations à long terme en Suisse

“…Once the complete model has been constructed, assessment of their validity is often needed to ensure that the predictions represent the most likely outcome in the real world (Yang et al, 2004). Moreover, although the behaviour of the functions for each stand variable within the model plays an important role in determining the overall outcome, the validity of each individual component does not guarantee the validity of the overall outcome, which is usually considered more important in practice.…”

“…The overall model outcome must therefore also be evaluated. The only method that can be regarded as "true" validation involves the use of a new independent data set (Pretzsch et al, 2002;Yang et al, 2004) but the scarcity of such data forces the use of alternative approaches. The common method of splitting the data set in two portions does not provide additional information (Huang et al, 2003) and neither is recommended from the point of view of parameter estimation.…”

“…Moreover, other techniques such as double cross-validation or statistical testing provide very limited information about the predictive ability of the models (Kozak and Kozak, 2003;Yang et al, 2004). Therefore, we decided to defer model validation until a new data set is available for assessing the quality of the predictions.…”

A stochastic height-diameter model for maritime pine ecoregions in Galicia (northwestern Spain)

“…According to [18,32,59], the final estimation of the model parameters should come from the entire data set because the estimations obtained with this approach will be more precise than those obtained from the model fitted from the split data set. Other alternatives of validation, such as cross-validation, do not provide any additional information compared with the respective statistics obtained directly from the model fitted with the entire data set [26,56].…”