Evaluating Research beyond Scientific ImpactHow to Include Criteria for Productive Interactions and Impact on Practice and Society

Abstract

Currently, established research evaluation focuses on scientific impact – that is, the impact of research on science itself. We discuss extending research evaluation to cover productive interactions and the impact of research on practice and society. The results are based on interviews with scientists from (organic) agriculture and a review of the literature on broader/social/societal impact assessment and the evaluation of interdisciplinary and transdisciplinary research. There is broad agreement about what activities and impacts of research are relevant for such an evaluation. However, the extension of research evaluation is hampered by a lack of easily usable data. To reduce the effort involved in data collection, the usability of existing documentation procedures (e.g., proposals and reports for research funding) needs to be increased. We propose a structured database for the evaluation of scientists, projects, programmes and institutions, one that will require little additional effort beyond existing reporting require ments.

Characteristics, emerging needs, and challenges of transdisciplinary sustainability science: experiences from the German Social-Ecological Research Program

“…It involves a http://www.ecologyandsociety.org/vol20/iss3/art13/ systematic, data-based procedure of description and assessment in which the people evaluating are the same ones who completed the activities evaluated (Stockmann and Meyer 2014). The set of evaluation criteria must be transparent, feasible, and flexible, depending on the goals of the evaluation (Defila and Di Giulio 1999, Klein 2008, Wolf et al 2013. Evaluation principles, characteristics, and criteria have been specifically proposed for TSS (Defila and Di Giulio 1999, Bergmann et al 2005, Wickson et al 2006, Klein 2008, Lang et al 2012.…”

Examining the Factors Behind the Success and Sustainability of China’s Creative Research Group: An Extension of the Teamwork Quality Model

“…These are important indices that have been emphasized by the NCSTE and NSFC, and have been applied in the evaluation of science and technology. While many scholars are developing new ways to measure the academic impact of scholars or teams [29,30], they are also constantly expanding their evaluation of the comprehensive impacts of scholars, teams and scientific research results [31,32]. Wolf, Lindenthal and many scientists believed that the existing research evaluation focused on the scientific impact of research teams and research results, and should gradually extend the research evaluation to the comprehensive evaluation of scientists and their teams, taking into account the comprehensive impact of their activities and achievements on economic and social aspects [31].…”

“…While many scholars are developing new ways to measure the academic impact of scholars or teams [29,30], they are also constantly expanding their evaluation of the comprehensive impacts of scholars, teams and scientific research results [31,32]. Wolf, Lindenthal and many scientists believed that the existing research evaluation focused on the scientific impact of research teams and research results, and should gradually extend the research evaluation to the comprehensive evaluation of scientists and their teams, taking into account the comprehensive impact of their activities and achievements on economic and social aspects [31]. Therefore, combined with the existing requirements for the evaluation of the comprehensive impact of CRG, and the research of relevant scholars, the theoretical model of TWQ is expanded here to include relevant elements of the SSCRG, which includes three variables in this paper: team performance, personal success, and comprehensive impacts (mainly, scientific and technological, economic, and social impacts.…”

“…Therefore, combined with the existing requirements for the evaluation of the comprehensive impact of CRG, and the research of relevant scholars, the theoretical model of TWQ is expanded here to include relevant elements of the SSCRG, which includes three variables in this paper: team performance, personal success, and comprehensive impacts (mainly, scientific and technological, economic, and social impacts. The SSCRG means the staged completion of the innovative project, the excellent evaluation of the project, the continuous operation of the team, the guarantee and possibility of the interests and benefits of team members, and the potential value and likelihood of continuous innovation, continuous growth and transition of the team in the future [4,8,[12][13][14][15][16][17]23,26,31]. The criteria for our judgment and evaluation are whether the CRG has successfully completed their research project; achieved their research goals and the cultivation of talents and teams; whether the individual members of the team have grown and attained greater motivation as researchers; and whether the team has made an important impact on academia and beyond.…”

Research Series Review for Transdisciplinarity Assessment—Validation with Sustainable Consumption and Production Research

“…TD research can be a promising way to gain insights to enhance implementation of SCP, which tackles societal problems often requiring multiple disciplines, as indicated by an advanced work [13]. However, a practical yet science-based procedure for assessing research planned or performed from the TD research perspective is still debated [14][15][16].…”