Assessing the quality of land system models: moving from valibration to evaludation

Abstract

Reviews suggest that evaluation of land system models is largely inadequate, with undue reliance on a vague concept of validation. Efforts to improve and standardise evaluation practices have so far had limited effect. In this article we examine the issues surrounding land system model evaluation and consider the relevance of the TRACE framework for environmental model documentation. In doing so, we discuss the application of a comprehensive range of evaluation procedures to existing models, and the value of each specific procedure. We develop a tiered checklist for going beyond what seems to be a common practice of ‘valibration’ (the repeated variation of model parameter values to achieve agreement with data) to achieving ‘evaludation’ (the rigorous, broad-based assessment of model quality and validity). We propose the Land Use Change – TRACE (LUC-TRACE) model evaludation protocol and argue that engagement with a comprehensive protocol of this kind (even if not this particular one) is valuable in ensuring that land system model results are interpreted appropriately. We also suggest that the main benefit of such formalised structures is to assist the process of critical thinking about model utility, and that the variety of legitimate modelling approaches precludes universal tests of whether a model is ‘valid’. Evaludation is therefore a detailed and subjective process requiring the sustained intellectual engagement of model developers and users.