Evaluating the integration of strategic priorities within a complex research-for-development funding program

Bleecker, Leah; Sauveplane-Stirling, Victoria; Ruggiero, Erica Di; Sellen, Daniel

doi:10.1016/j.evalprogplan.2021.102009

Cited by 2 publications

(11 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most assessments covered public funders (33, 75.0%). One article featured an academic funder [ 56 ], three articles featured charity funders [ 33 , 57 , 58 ], four articles featured NGO funders [ 55 , 59 – 61 ], five articles featured a funding portfolio of mixed funder types [ 34 , 35 , 62 – 64 ], and two articles did not specify a funder type for the funding portfolio they assessed [ 51 , 65 ].…”

Section: Resultsmentioning

confidence: 99%

“…Ayenew et al at the United States Centers for Disease Control and Prevention (CDC) examined the role their Achieving Public Health Impact through Research (APHIR) contract mechanism played in helping individual projects achieve their intended project outcomes [ 86 ], and Javorka et al sought to determine the added value of a concordat (joint funding agreement) between the United Kingdom MRC and Department for International Development (DFID) during 2013–2018 [ 77 ]. Further, Bleecker et al and Javorka et al were interested in examining how their funding programmes not only created impact but supported the dissemination of the resulting research [ 55 , 77 ].…”

Section: Resultsmentioning

confidence: 99%

“…Most included articles (33, 75.0%) featured impact assessments that dealt with funding portfolios based at the project or grant level. Articles addressing other types of research funding included the following: two articles were large ROI studies which looked at all research funding in a particular sector (from multiple funders) by total research spend across funders, not at outcomes generated at the individual project level by funder [ 35 , 62 ]; one article examined the impact of Cochrane reviews themselves (arising from previous NIHR project funding) [ 70 ]; two articles examined prepared case studies of funded projects [ 73 , 87 ]; one article examined the collective impact of all Marie Curie palliative and end-of-life care funding, including research projects but also research centres, programme leads, hospices and fellowship funding [ 58 ]; one article examined the scientometric impact of researchers funded by European Research Council projects [ 76 ]; two studies examined the impact of the NIH CTSA Program hubs, where the unit of analysis was a CTSA hub (which necessarily meant they had been the recipient of an NIH-funded award) [ 52 , 53 ]; one study examined the impact of European biomedical research infrastructures in the fight against COVID-19 [ 54 ], where the unit of analysis was funded infrastructures but the project/award mechanisms were not well specified; and two studies did not provide enough information to determine the unit of portfolio being assessed [ 55 , 88 ].…”

Section: Resultsmentioning

confidence: 99%

“…Five articles, all from public funders [ 67 , 70 , 74 , 89 , 90 ], developed their own (unnamed) impact framework for their analysis, often adapted from a leading framework such as the Payback [ 23 ] or CAHS [ 24 ] frameworks. Another five articles (spanning public, NGO and mixed funders) used a more general methodology/framework that differed from the traditional impact frameworks but had been described elsewhere in the literature: Complexity Theory [ 55 ], Context Matters Framework [ 53 ], Contribution Mapping [ 63 ], Di Ruggiero et al field-building framework [ 61 ] and a general intervention mixed-methods framework [ 52 ].…”

Section: Resultsmentioning

confidence: 99%

“…After an extensive review, 99.9% (26,122) of articles found via database searching and 76.4% [ 55 ] of grey literature articles were excluded. Articles may have been excluded because they were not related to biomedical health, were not related to a funding portfolio, did not perform a RIA, or performed a bibliometrics-only-focused assessment.…”

Section: Methodsmentioning

confidence: 99%

See 4 more Smart Citations

What funders are doing to assess the impact of their investments in health and biomedical research

Abudu

Oliver

Boaz

2022

Health Res Policy Sys

View full text Add to dashboard Cite

As pressures to maximize research funding grow, biomedical research funders are increasingly tasked with demonstrating the long-term and real-world impacts of their funded research investments. Over the past three decades, research impact assessments (RIA) have emerged as an important tool for analysing the impacts of research by incorporating logic models, frameworks and indicators to track measures of knowledge production, capacity-building, development of research products, adoption of research into clinical guidelines and policies, and the realization of health, economic and social benefits. While there are currently several models for RIA within the literature, less attention has been paid to how funders can practically select and implement a RIA model to demonstrate the impacts of their own research portfolios. In this paper, a literature review was performed to understand (1) which research funders have performed RIAs of their research portfolios to date; (2) how funders have designed their assessments, including the models and tools they have used; (3) what challenges to and facilitators of success have funders found when adopting the RIA model to their own portfolio; and (4) who participates in the assessments. Forty-four papers from both published and grey literature were found to meet the review criteria and were examined in detail. There is a growing culture of RIA among funders, and included papers spanned a diverse set of funders from 10 countries or regions. Over half of funders (59.1%) used a framework to conduct their assessment, and a variety of methods for collecting impact data were reported. Issues of methodological rigour were observed across studies in the review, and this was related to numerous challenges funders faced in designing timely RIAs with quality impact data. Over a third of articles (36.4%) included input from stakeholders, yet only one article reported surveying patients and members of the public as part of the assessment. To advance RIA among funders, we offer several recommendations for increasing the methodological rigour of RIAs and suggestions for future research, and call for a careful reflection of the voices needed in an impact assessment to ensure that RIAs are having a meaningful impact on patients and the public.

show abstract