Energy economy optimization (EEO) models employ formal search techniques to explore the future decision space over several decades in order to deliver policy-relevant insights. EEO models are a critical tool for decision-makers who must make near-term decisions with long-term effects in the face of large future uncertainties. While the number of model-based analyses proliferates, insufficient attention is paid to transparency in model development and application. Given the complex, dataintensive nature of EEO models and the general lack of access to source code and data, many of the assumptions underlying model-based analysis are hidden from external observers. This paper discusses the simplifications and subjective judgments involved in the model building process, which cannot be fully articulated in journal papers, reports, or model documentation. In addition, we argue that for all practical purposes, EEO model-based insights cannot be validated through comparison to real world outcomes. As a result, modelers are left without credible metrics to assess a model's ability to deliver reliable insight. We assert that EEO models should be discoverable through interrogation of publicly available source code and data. In addition, third parties should be able to run a specific model instance in order to independently verify published results. Yet a review of twelve EEO models suggests that in most cases, replication of model results is currently impossible. We provide several recommendations to help develop and sustain a software framework for repeatable model analysis.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.