Increasing policy demand for realist evaluations of research and capacity-building programmes reflects a recognition of the management, governance and impact gains that can result from evaluation. However, the evidence base on how to successfully implement realist evaluations of complex interventions in international development efforts is scarce. We know little about the associated merits, limitations and ways to mitigate challenges. There is a need for reflective work which considers the methodology in context. This paper shares learning from the experience of conducting a realist, theory-of-change driven evaluation of the African Institutions Initiative, a Wellcome Trust funded programme which aimed to build sustainable health research capacity in Africa at institutional and network levels, across seven research consortia.We reflect on the key challenges experienced throughout the evaluation and recommend ways of managing them, highlight opportunities and critical success factors associated with this evaluation approach, as well as elaborate on alternative evaluation approaches.

No abstract

No abstract

Purpose Assessing the impact of research requires an approach that is sensitive both to the context of the research and the perspective of the stakeholders trying to understand its benefits. Here, the authors report on a pilot that applied such an approach to research conducted at the Collaborative Center for Health Equity (CCHE) of the University of Wisconsin School of Medicine and Public Health. Method The pilot assessed the academic impact of CCHE’s work; the networks between CCHE and community partners; and the reach of CCHE’s programs, including an attempt to estimate return on investment (ROI). Data included bibliometrics, findings from a stakeholder survey and in-depth interviews, and financial figures. Results The pilot illustrated how CCHE programs increase the capacity of community partners to advocate for their communities and engage with researchers to ensure research benefits the community. The results illustrate the reach of CCHE’s programs into the community. The authors produced an estimate of the ROI for one CCHE program targeting childhood obesity, and values ranged from negative to positive. Conclusions The authors experienced challenges using novel assessment techniques at a small scale including the lack of comparator groups and the scarcity of cost data for estimating ROI. This pilot demonstrated the value of research from a variety of perspectives – from academic to community. It illustrates how metrics beyond grant income and publications can capture the outputs of an academic health center in a way that may better align with the aims of the center and stakeholders.