Evaluating Research Impact: The Development of a Research for Impact Tool

Abstract

IntroductionThis paper examines the process of developing a Research for Impact Tool in the contexts of general fiscal constraint, increased competition for funding, perennial concerns about the over-researching of Aboriginal and Torres Strait Islander issues without demonstrable benefits as well as conceptual and methodological difficulties of evaluating research impact. The aim is to highlight the challenges and opportunities involved in evaluating research impact to serve as resource for potential users of the research for impact tool and others interested in assessing the impact of research.Materials and methodsA combination of literature reviews, workshops with researchers, and reflections by project team members and partners using participatory snowball techniques.ResultsAssessing research impact is perceived to be difficult, akin to the so-called “wicked problem,” but not impossible. Heuristic and collaborative approach to research that takes the expectations of research users, research participants and the funders of research offers a pragmatic solution to evaluating research impact. The logic of the proposed Research for Impact Tool is based on the understanding that the value of research is to create evidence and/or products to support smarter decisions so as to improve the human condition. Research is, therefore, of limited value unless the evidence created is used to make smarter decisions for the betterment of society. A practical way of approaching research impact is, therefore, to start with the decisions confronting decision makers whether they are government policymakers, industry, professional practitioners, or households and the extent to which the research supports them to make smarter policy and practice decisions and the knock-on consequences of doing so. Embedded at each step in the impact planning and tracking process is the need for appropriate mix of expertise, capacity enhancement, and collaborative participatory learning-by-doing approaches.DiscussionThe tool was developed in the context of Aboriginal and Torres Strait Islander research but the basic idea that the way to assess research impact is to start upfront with the information needs of decisions makers is equally applicable to research in other settings, both applied (horizontal) and basic (vertical) research. The tool will be further tested and evaluated with researchers over the next 2 years (2016/17). The decision by the Australian Government to include ‘industry engagement’ and ‘impact’ as additions to the Excellence in Research for Australia (ERA) quality measures from 2018 makes the Research for Impact Tool a timely development. The wider challenge is to engage with major Australian research funding agencies to ensure consistent alignment and approaches across research users, communities, and funders in evaluating impact.

Measuring research impact in medical research institutes: a qualitative study of the attitudes and opinions of Australian medical research institutes towards research impact assessment frameworks

“…The majority of participants were supportive of the rationale, but raised questions regarding the mechanisms through which prospective assessment would be implemented. The opportunity that this approach presents for impact assessment merits further investigation, including the initiatives pursued by Searles et al [ 13 ], Graham et al [ 24 ], Tsey et al [ 25 ], Trochim et al [ 26 ], Herbert et al [ 27 ] and Greenhalgh et al [ 28 ], as it potentially holds implications for the methods and the systems required to support this approach.…”

An “All Teach, All Learn” Approach to Research Capacity Strengthening in Indigenous Primary Health Care Continuous Quality Improvement

“…acknowledge that Aboriginal and Torres Strait Islander autonomy and participation are core to ethical research in Indigenous studies ( 41 ). Aboriginal and Torres Strait Islander people have been long calling for reform of research ( 7 , 26 , 33 , 42 ), for which it is essential that research relating to Indigenous people is carried out on their terms, led and directed by Indigenous people ( 43 ).…”

“…Strengthening networking and partnerships for CQI-RCS are key for transfer and implementation of innovations ( 58 ), and accountability and research priorities that benefit Indigenous Australians ( 7 ). Indigenous ownership and stakeholder relationships from the outset of research enhance the likelihood of research relevance and thus translation and benefit ( 43 ). Allowing sufficient time for meaningful relationship building is essential for quality research in this space and is a RCS activity itself as it enables respectful engagement with Indigenous knowledge and perspectives ( 26 , 34 , 59 ).…”

The Development of a Survey Instrument to Assess Aboriginal and Torres Strait Islander Students’ Resilience and Risk for Self-Harm

“…Based on the work of Searles et al (2016), a program logic model was developed to identify the key pathway between TSS student support and potential outcomes and the expected attributable benefit of the TSS/research approach ( Figure 2). Research benefit was understood as the establishment or enhancement of capacities, opportunities or outcomes that advance the interests of Indigenous people and that are valued by them (Bainbridge et al, 2015;Tsey et al, 2016).…”