A bricolage-style exploratory scenario analysis to manage uncertainty in socio-environmental systems modeling: investigating integrated water management options

Abstract

Exploratory analysis, while useful in assessing the implications of model assumptions under large uncertainty, is considered at best a semi-structured activity. There is no algorithmic way for performing exploratory analysis and the existing canonical techniques have their own limitations. To overcome this, we advocate a bricolage-style exploratory scenario analysis, which can be crafted by pragmatically and strategically combining different methods and practices. Our argument is illustrated using a case study in integrated water management in the Murray-Darling Basin, Australia. Scenario ensembles are generated to investigate potential policy innovations, climate and crop market conditions, as well as the effects of uncertainties in model components and parameters. Visualizations, regression trees and marginal effect analyses are exploited to make sense of the ensemble of scenarios. The analysis includes identifying patterns within a scenario ensemble, by visualizing initial hypotheses that are informed by prior knowledge, as well as by visualizing new hypotheses based on identified influential variables. Context-specific relationships are explored by analyzing which values of drivers and management options influence outcomes. Synthesis is achieved by identifying context-specific solutions to consider as part of policy design. The process of analysis is cast as a process of finding patterns and formulating questions within the ensemble of scenarios that merit further examination, allowing end-users to make the decision as to what underlying assumptions should be accepted, and whether uncertainties have been sufficiently explored. This approach is especially advantageous when the precise intentions of management are still subject to deliberations. By describing the reasoning and steps behind a bricolage-style exploratory analysis, we hope to raise awareness of the value of sharing this kind of (common but not often documented) analysis process, and motivate further work to improve sharing of know-how about bricolage in practice.

Identifying Minimum Information Requirements to Improve Integrated Modeling Capabilities: Lessons Learned From Dynamic Adaptive Policy Pathways

“…This is aligned with the idea of incremental learning, notably emphasized in the ML field, and identifying fit-for-purpose solutions for solving incremental learning tasks (Luo et al, 2020). Such incremental learning assumes ongoing adaptation of models to learning (Luo et al, 2020) and allows for rapid feedback when evaluating planning assumptions and the identification of potential areas to explore prior to considering further investments (Fu et al, 2020;Zare et al, 2020). In addition, it assumes a flexible structure when considering solutions, with implications for model interoperability (Madni and Sievers, 2014), especially when considering implementationbased solutions.…”

“…In that case, other approaches could be considered such as exploring uncertain boundary conditions using a semi-probabilistic model to map conditions of failure and ATP occurrence (Raso et al, 2019a). Another solution could be to use "bricolage-type" approaches and various sets of visualization methods to acknowledge and map uncertainty (Fu et al, 2015(Fu et al, , 2020. Future research will explore how these approaches could better contribute to the refinement of ATP conditions and their expected timing and implications for action robustness and associated adaptation pathways.…”

“…Signposts and conditions of success/failure are not expected to be transferable beyond closely similar case studies. However, principles discussed and associated with the identification and use of signposts may be transferable, including criteria to select relevant signposts (Raso et al, 2019b), procedures to align with users' information requirements through reflection, discussion and deliberation (Bauler, 2012;Peppard and Campbell, 2014;Zare et al, 2020), consideration of different dimensions of robustness (Kalra et al, 2014;Lawrence et al, 2019), and approaches to handle iteration and emergent events (Peppard and Campbell, 2014;Fu et al, 2020).…”

Modeling Water Quality in Watersheds: From Here to the Next Generation

“…Furthermore, there is still a large gap between the skill set of most modelers and the skill set required to effectively use high-performance computing systems for robust experimental-type simulations (e.g., model intercomparison studies, calibrations, sensitivity analyses, and uncertainty analyses). Better tools are needed to support the development, sharing, and reuse of modeling and analysis workflows, supported by better publishing of the workflows ( Fu et al, 2020 ). For example, the recently developed open source Mobius model building system provides a virtual environmental laboratory for practitioners with little programming experience to quickly develop and evaluate watershed water quality models, making uncertainty analysis more accessible to model users ( Norling et al, 2020 ).…”