Critical Evaluation of Models and Their Parameters

Abstract

Some form of critical evaluatory procedure for models is needed to maintain the integrity of modeling and to ensure that the increasingly widespread use of models does not result in the propagation of misleading information. The term validation must be used with the clear understanding that no model can be validated in the sense that it has been unequivocally justified. All that can be achieved is to show how small the probability is that the model has been refuted. Whether this probability is acceptable is a subjective decision. The type of statistical test that is appropriate depends on the quality of the data against which the model is tested. Using a procedure that compares the sums of squares that result from the model not fitting the data with the sums of squares due to error in the data, gives a stringent test, but it requires the replication of measurements. Rigor is as important in evaluating parameters as it is in testing models. Direct measurement is the best option, but where a parameter has to be obtained by fitting, the statistical procedures used for validation are appropriate. In general, the further the data used for parameterization are removed from the data to be simulated the better. Problems can arise in both parameterization and validation if the model is nonlinear regarding its parameters, and the latter have appreciable variances. Parameterization and validation become more difficult as the complexity of the model or the scale at which it is used increase.

A multi-source satellite data approach for modelling Lake Turkana water level: calibration and validation using satellite altimetry data

“…Uncertainty in hydrologic models includes uncertainties in (a) the structure of the model, (b) the model parameters/input data, and (c) the solution of the model (Addiscott et al, 1995). In most hydrologic models, when using satellite data such as the LLM approach, major uncertainties in the model outputs can be attributed to the model parameters or input data.…”

Transferability and scaling of soil total carbon prediction models in Florida

“…Addiscott, Smith and Bradbury (1995) pointed out that an increase in parameter variance may cause problems by interacting with the linearity / non-linearity in the process represented by the model. McBratney (1998) and Grunwald et al (2011) asserted that an increase in the variance of soil attributes can impact the model building process, transferability, and scalability of soil properties.…”

Validation of the exposure assessment for veterinary medicinal products

“…The first objective of this paper is to validate existing screening level exposure models for the risk assessment of veterinary medicines spread in manure that have been presented previously in this journal by . Addiscot et al (1995) stated that some form of critical evaluation procedure is essential to maintain the integrity of modelling and to ensure that the use of models by regulators does not result in the propagation of misleading information. Validation is used here in the meaning of establishing whether the model is 'well founded and applicable' (Addiscot and Wagenet, 1985, Dee, 1995and Addiscot et al, 1995.…”

“…Addiscot et al (1995) stated that some form of critical evaluation procedure is essential to maintain the integrity of modelling and to ensure that the use of models by regulators does not result in the propagation of misleading information. Validation is used here in the meaning of establishing whether the model is 'well founded and applicable' (Addiscot and Wagenet, 1985, Dee, 1995and Addiscot et al, 1995. Screening level models do not intend to represent reality accurately, but to provide rapid predictions of the potential environmental fate of a compound (Tarazona et al, 2003).…”