A framework for characterising and evaluating the effectiveness of environmental modelling

Hamilton, Serena H.; Fu, Baihua; Guillaume, Joseph H. A.; Badham, Jennifer; Elsawah, Sondoss; Gober, Patricia; Hunt, Randall J.; Iwanaga, Takuya; Jakeman, Anthony J.; Ames, Daniel P.; Curtis, Allan; Hill, Mary C.; Pierce, Suzanne A.; Zare, Fateme

doi:10.1016/j.envsoft.2019.04.008

Cited by 66 publications

(92 citation statements)

References 121 publications

(156 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Despite these conceptual guidelines, it remains challenging for modelers to understand, from the outset, how modeling outputs can and will be used in practice. As stated by Hamilton et al (: 21), “water resource models are an obvious way for [scientists and decision‐makers] to work together, but uneven power relations and the institutional differences between academic and public sector employment can stymie their role in mediating interests of the two groups.” In this context, our research seeks to understand the role that hydrologic modeling (and coupled monitoring) can play in supporting real‐world watershed management programs.…”

Section: Introductionmentioning

confidence: 99%

Who Are we Measuring and Modeling for? Supporting Multilevel Decision‐Making in Watershed Management

Bremer

Hamel²,

Ponette‐González

et al. 2020

Water Resources Research

View full text Add to dashboard Cite

As watershed management programs have become more common globally, so have efforts to support these initiatives through hydrologic modeling and monitoring. However, these efforts are often guided by oversimplified assumptions of how management programs work and the quantity, quality, and type of information needed to support their planning, implementation, and evaluation. Semi-structured interviews and focus groups with project managers, funders, and participants in three watershed management programs in the Atlantic Forest of Brazil revealed a range of hydrologic modeling and monitoring needs of watershed management programs. We identify five opportunities for hydrologic information to support overlapping management contexts: (1) inspire action and support, (2) inform investment decisions, (3) engage with potential participants, (4) prioritize location and types of activities at regional to national scales, and (5) evaluate program success. Within these opportunities, understanding who will use the information generated and how they will do so is critical to increasing the salience, credibility, and legitimacy of modeling efforts. Hydrologic modeling and monitoring play a small but critical role in the larger context of program conceptualization, design, implementation, and evaluation; grounding these efforts in local contexts supports watershed management projects in relevant and effective ways.Plain Language Summary Active land management for a variety of benefits, including sustaining and enhancing clean and ample water supplies, is becoming more common worldwide. To achieve these ends, watershed management programs need the support of hydrologic data and models, but promising efforts by the hydrologic community often go unused when the information program managers need is not well matched to modeling efforts. We interviewed a wide range of participants in water management programs in Brazil and found five key areas where modeling and monitoring can support these programs: (1) inspire action and support, (2) inform investment decisions, (3) engage with potential participants, (4) prioritize location and types of activities at regional to national scales, and (5) evaluate program success. Our study emphasizes the importance of focusing on who will use modeling results and tailoring efforts to meet these needs. When grounded in real-world contexts, hydrologic monitoring and modeling can play a small but critical role in supporting sustainable watershed management

show abstract

Section: Introductionmentioning

confidence: 99%

Who Are we Measuring and Modeling for? Supporting Multilevel Decision‐Making in Watershed Management

Bremer

Hamel²,

Ponette‐González

et al. 2020

Water Resources Research

View full text Add to dashboard Cite

show abstract

“…This evaluation extends beyond the technical performance of the SoS model ( Bennett et al, 2013 ) to consider outcomes of modeling as a social process. Success of a model depends on the beliefs and expectations of the intended users and in their satisfaction with the model and its results ( Hamilton et al, 2019 ). It may also depend on the biases and beliefs of the model creators ( Glynn et al, 2017 ) and in an alignment of expectations between creators and users ( Sterling et al, 2019 ).…”

Section: Scale Issues To Considermentioning

confidence: 99%

“…It may also depend on the biases and beliefs of the model creators ( Glynn et al, 2017 ) and in an alignment of expectations between creators and users ( Sterling et al, 2019 ). The suitability of the success criteria is dependent on the context of the project, including not only the model purpose, but also the characteristics of the problem, such as its complexity and the resources that were available ( Hamilton et al, 2019 ).…”

Section: Scale Issues To Considermentioning

confidence: 99%

Socio-technical scales in socio-environmental modeling: Managing a system-of-systems modeling approach

Iwanaga

Wang

Hamilton

et al. 2021

Environmental Modelling & Software

Self Cite

View full text Add to dashboard Cite

System-of-systems approaches for integrated assessments have become prevalent in recent years. Such approaches integrate a variety of models from different disciplines and modeling paradigms to represent a socio-environmental (or social-ecological) system aiming to holistically inform policy and decision-making processes. Central to the system-of-systems approaches is the representation of systems in a multi-tier framework with nested scales. Current modeling paradigms, however, have disciplinary-specific lineage, leading to inconsistencies in the conceptualization and integration of socio-environmental systems. In this paper, a multidisciplinary team of researchers, from engineering, natural and social sciences, have come together to detail socio-technical practices and challenges that arise in the consideration of scale throughout the socio-environmental modeling process. We identify key paths forward, focused on explicit consideration of scale and uncertainty, strengthening interdisciplinary communication, and improvement of the documentation process. We call for a grand vision (and commensurate funding) for holistic system-of-systems research that engages researchers, stakeholders, and policy makers in a multi-tiered process for the co-creation of knowledge and solutions to major socio-environmental problems.

show abstract

“…Using data products prepared and validated by experts can help in building confidence in the analysis results. With reference to the evaluation framework for environmental modelling developed by Hamilton et al (2019), avoiding rushed modelling by inexperienced modellers can improve the confidence in several project-level elements of that framework -1) efficiency by reducing time needed to produce estimates, 2) credibility by using outputs from professional hydrologists, 3)…”

Section: Planned Future Developmentsmentioning

confidence: 99%

“…Using a model evaluation framework (e.g. Hamilton et al 2019), an https://doi.org/10.5194/gmd-2020-276 Preprint. Discussion started: 26 October 2020 c Author(s) 2020.…”

Section: Introductionmentioning

confidence: 99%

Hydrostreamer v1.0 – improved streamflow predictions for local applications from an ensemble of downscaled global runoff products

Kallio¹,

Guillaume²,

Virkki³

et al. 2020

Preprint

View full text Add to dashboard Cite

Abstract. An increasing number of different types of hydrological, land surface, and rainfall-runoff models exist to estimate streamflow in river networks. Results from various model runs from global to local scale are readily available online. However, the usability of these products is often limited, as they often come aggregated in spatial units which are not compatible with the desired analysis purpose. We present here an R package, a software library hydrostreamer v1.0 which aims to improve the usability of existing runoff products by addressing the Modifiable Area Unit Problem, and allows non-experts with little knowledge of hydrology-specific modelling issues and methods to use them for their analyses. Hydrostreamer workflow includes 1) interpolation from source zones to target zones, 2) river routing, and 3) data assimilation via model averaging, given multiple input runoff and observation data. The software implements advanced areal interpolation methods and area-to-line interpolation not available in other products, and is the first R package to provide vector-based routing. Hydrostreamer is kept as simple as possible – intuitive with minimal data requirements – and minimizes need for calibration. We tested the performance of hydrostreamer by downscaling freely available coarse-resolution global runoff products from the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP) in an application in 3S Basin in Southeast Asia. Results are compared to observed discharges as well as two benchmark streamflow data products, finding comparable or improved performance. Hydrostreamer v1.0 is open source and is available from http://github.com/mkkallio/hydrostreamer/ under MIT licence.

show abstract

A framework for characterising and evaluating the effectiveness of environmental modelling

Cited by 66 publications

References 121 publications

Who Are we Measuring and Modeling for? Supporting Multilevel Decision‐Making in Watershed Management

Who Are we Measuring and Modeling for? Supporting Multilevel Decision‐Making in Watershed Management

Socio-technical scales in socio-environmental modeling: Managing a system-of-systems modeling approach

Hydrostreamer v1.0 – improved streamflow predictions for local applications from an ensemble of downscaled global runoff products

Contact Info

Product

Resources

About