Application of a variance‐based sensitivity analysis method to the Biomass Scenario Learning Model

Abstract

Variance-based sensitivity analysis can provide a comprehensive understanding of the input factors that drive model behavior, complementing more traditional system dynamics methods with quantitative metrics. This paper presents the methodology of a variance-based sensitivity analysis of the Biomass Scenario Learning Model, a published STELLA model of interactions among investment, production, and learning in an emerging competitive industry. We document the methodology requirements, interpretations, and constraints, and compute estimated sensitivity indices and their uncertainties. We show that application of variance-based sensitivity analysis to the model allows us to test for non-additivity, identify influential and interactive variables, and confirm model formulation. To enable use of this type of sensitivity analysis in other system dynamics models, we provide this study's R code, annotated to facilitate adaptation to other studies. A related paper describes application of these techniques to the much larger Biomass Scenario Model. has twenty five years of experience with infrastructure, energy, and transportation modeling, simulation, and analysis.Daniel Inman, a PhD soil scientist, has over ten years of experience in bioenergy feedstocks, modeling, and advanced statistics.Steve Peterson is an independent consultant and a Senior Lecturer at the Thayer School of Engineering at Dartmouth College. Steve has been involved in the teaching and application of dynamic modeling approaches since the early 1980s.Additional supporting information may be found online in the Supporting Information section at the end of the article.Appendix S1 Model documentation and annotated R code. P. Jadun et al.: Variance-based Sensitivity Analysis of the Biomass Scenario Learning Model 335

Lessons from a large‐scale systems dynamics modeling project: the example of the biomass scenario model

“…Positive feedbacks around investment, production and industrial learning can drive responses, depending on initial maturity, nth plant attributes such as capital cost or process yield, and progress ratios that govern the speed by which production translates into increases in maturity. From Jadun et al (2017) [Color figure can be viewed at wileyonlinelibrary.com] Donella Meadows (2008) describes coupled positive feedbacks such these as a "systems trap" because they can lead to "lock-in" responses. The NREL team used a highly simplified version of the BSM conversion modules to explore the dynamics associated with these coupled feedbacks (Jadun et al, 2017).…”

“…From Jadun et al (2017) [Color figure can be viewed at wileyonlinelibrary.com] Donella Meadows (2008) describes coupled positive feedbacks such these as a "systems trap" because they can lead to "lock-in" responses. The NREL team used a highly simplified version of the BSM conversion modules to explore the dynamics associated with these coupled feedbacks (Jadun et al, 2017). In its baseline behavior ( Figure 5), three hypothetical pathways with different initial maturities, process yields and capital costs compete for investment dollars in an environment that offers incentives for production and for start-up investment.…”

“…Pathway C has higher initial maturity, but lower process yields and higher capital cost than A and B. Pathway B has higher initial maturity, but lower process yields than A. Adapted fromJadun et al (2017, Figure 3)[Color figure can be viewed at wileyonlinelibrary.com]…”

Using R libraries to facilitate sensitivity analysis and to calibrate system dynamics models

“…These methodological advances in data science and artificial intelligence now provide system dynamics modelers with opportunities to apply robust computational methods to model building, and, interestingly, simulation modeling is increasingly viewed as a methodological asset to the field of data science (Neuwirth, ). Recent examples of computational advances in system dynamics include: a presentation of analytical tools for model building, estimation and analysis (Rahmandad et al ., ); methods involving sensitivity analysis (Hekimoğlu and Barlas, ; Walrave, ; Jadun et al ., ); and machine learning approaches (Kwakkel, ).…”

Input and output data analysis for system dynamics modelling using the tidyverse libraries of R

“…These results must be interpreted and analysed as part of the model‐building and policy analysis process. Third, simulation methods such as sensitivity analysis (Hekimoğlu and Barlas, ; Walrave, ; Jadun et al, ) generate large datasets that need to be processed for further analysis—for example, techniques such as statistical screening (Ford and Flynn, ; Taylor et al, ; Yasaman and Ford, ). Therefore, in the context of these data‐intensive modelling processes, there are opportunities for system dynamics modellers to leverage complementary data exploration technologies such as R (Duggan, ).…”