A checklist for model credibility, salience, and legitimacy to improve information transfer in environmental policy assessments

Abstract

a b s t r a c tModelers involved in environmental policy assessments are commonly confronted with the lack of uptake of model output by policy actors. Actors have different expectations of models, condensed into three quality criteria: credibility, salience, and legitimacy. The fulfilment of quality criteria is also dynamic as expectations vary, change, and possibly counteract each other. We present a checklist for modelers involved in model-based assessments that is aimed at the identification and monitoring of issues, limitations and trade-offs regarding model quality criteria. It draws upon the literature of integrated assessments as well as case study analysis of environmental policy assessments for the Dutch government, based on expert interviews and embedded experience. The checklist is intended to be consulted during assessments; its application may result in greater awareness among modelers involved in assessments regarding model quality criteria, and may positively affect the uptake of model-based knowledge from environmental policy assessments by policy actors.

Participatory Modeling Updates Expectations forIndividuals and Groups, Catalyzing BehaviorChange and Collective Action inWater‐Energy‐Food NexusGovernance

“…Second, model‐based decision support is an area ripe for innovation in sustainability research (Basco‐Carrera et al, ; Gerritsen et al, ; van Voorn et al, ). New insights on how this form of knowledge production and use could translate to behavior change have potential to scale given its common usage.…”

A compiled project and open-source code to generate web-based forest modelling simulators

“…Gómez- García et al Computers and Electronics in Agriculture 147 (2018) 1-5 the first application of ForestMTIS, is a compilation of models from different modellers and a direct link with the end-users, contributing to direct transferring of research results and learning regarding sustainable forest management. Van Voorn et al (2016) pointed out three criteria for end-users to have interest in models which can also be extended to simulators: (i) credibility, i.e., if the model is considered an acceptable approximation of the modelled system, (ii) salience, i.e., if it can answer the questions that are raised, and (iii) legitimacy, i.e., when it represents fairly the views, values and concerns of involved stakeholders. In addition, modellers should be aware of four points (van Voorn et al, 2016): (a) the three criteria above cannot be considered equal but should be balanced, (b) the criteria can be counteracting, (c) the perception of credibility varies among modellers and end-users, and (d) the perception of the three criteria seem to be dynamic and may be path-dependent.…”

“…Van Voorn et al (2016) pointed out three criteria for end-users to have interest in models which can also be extended to simulators: (i) credibility, i.e., if the model is considered an acceptable approximation of the modelled system, (ii) salience, i.e., if it can answer the questions that are raised, and (iii) legitimacy, i.e., when it represents fairly the views, values and concerns of involved stakeholders. In addition, modellers should be aware of four points (van Voorn et al, 2016): (a) the three criteria above cannot be considered equal but should be balanced, (b) the criteria can be counteracting, (c) the perception of credibility varies among modellers and end-users, and (d) the perception of the three criteria seem to be dynamic and may be path-dependent. Therefore, models developed without feed-back or even involvement of end-users can be salient but not credible or legitimate.…”

Linking knowledge to action in ocean ecosystem management: The Ocean Modeling Forum

“…The challenge to link knowledge to action for effective natural resource management is now well recognized (Knight et al, 2008;Weaver et al, 2014;Clark et al, 2016b;Beier et al, 2017;Wall et al, 2017). Knowledge that is useful to management is salient (relevant to management needs), credible (authoritative and trusted), and legitimate (developed within a process that accounts for all stakeholder perspectives) (Cook et al, 2013;Kunseler et al, 2015;van Voorn et al, 2016). Recognition of the need to make science usable has led to efforts to improve science communication (Fischhoff and Scheufele, 2014, and references therein;Young et al, 2014), to explicitly engage in the process of knowledge co-production or collection (Jacobs et al, 2016;Reid et al, 2016), and to increase support for interdisciplinary research groups (Levin et al, 2016), but few of those efforts translate to management action.…”