Developing Evaluation Indicators to Improve the Process of Coproducing Usable Climate Science

Abstract

Resource managers and decision-makers are increasingly tasked with integrating climate change science into their decisions about resource management and policy development. This often requires climate scientists, resource managers, and decision-makers to work collaboratively throughout the research processes, an approach to knowledge development that is often called ''coproduction of knowledge.'' The goal of this paper is to synthesize the social science theory of coproduction of knowledge, the metrics currently used to evaluate usable or actionable science in several federal agencies, and insights from experienced climate researchers and program managers to develop a set of 45 indicators supporting an evaluation framework for coproduced usable climate science. Here the proposed indicators and results from two case studies that were used to test the indicators are presented, as well as lessons about the process of evaluating the coproduction of knowledge and collaboratively producing climate knowledge.

On the comparability of knowledge transfer activities – a case study at the German Baltic Sea Coast focusing regional climate services

“…Arnott et al (2016) describe different purposes of evaluation, which are either to decide about the existence or funding of a program or to improve their efficiency. Most recently, Wall et al (2017) has conducted a broad review of corresponding literature and synthesize the metrics currently used to evaluate usable or actionable climate science. It is suggested that six different components should be evaluated.…”

Evaluating co-creation of knowledge: from quality criteria and indicators to methods

“…Indicators in this sense refer to a unit to asses a specific state of something. Wall et al (2017) mix up what Figure 2. The process of co-creation (on the left) spreads up into three consecutive phases (Pohl and Hirsch Hadorn, 2007), each of them can be evaluated.…”

“…The few publications dealing with criteria in a concrete manner show an enormous pool (i.e. Bergmann et al, 2005;Wall et al, 2017). Nevertheless, it seems important to compile all of the ideas in one overall representation.…”

“…427, 428). Wall et al (2017) gave a comprehensive overview of the different concepts and terms in the area of transdisciplinary research worldwide.…”

“…Rafols et al (2012) showed how the bibliometric evaluation method might panelize inter-and transdisciplinary approaches. Meanwhile the need for new evaluation criteria and methodologies is absolutely agreed upon (Wolf et al, 2013;Jahn and Keil, 2015;Wall et al, 2017).…”