Evaluating a complex research capacity-building intervention: Reflections on an evaluation of the African Institutions Initiative

Abstract

Increasing policy demand for realist evaluations of research and capacity-building programmes reflects a recognition of the management, governance and impact gains that can result from evaluation. However, the evidence base on how to successfully implement realist evaluations of complex interventions in international development efforts is scarce. We know little about the associated merits, limitations and ways to mitigate challenges. There is a need for reflective work which considers the methodology in context. This paper shares learning from the experience of conducting a realist, theory-of-change driven evaluation of the African Institutions Initiative, a Wellcome Trust funded programme which aimed to build sustainable health research capacity in Africa at institutional and network levels, across seven research consortia.We reflect on the key challenges experienced throughout the evaluation and recommend ways of managing them, highlight opportunities and critical success factors associated with this evaluation approach, as well as elaborate on alternative evaluation approaches.

Strategies for developing sustainable health research capacity in low and middle-income countries: a prospective, qualitative study investigating the barriers and enablers to locally led clinical trial conduct in Ethiopia, Cameroon and Sri Lanka

“…This is supported by recent contributions to the literature from WHO-TDR and ESSENCE on Health Research that continue to view the issues raised in this paper as problematic 40 41 and practitioner calls for greater investment in research capacity building and its evaluation to support emerging research agendas. 21 42 …”

Strengthening Mentoring in Low- and Middle-Income Countries to Advance Global Health Research: An Overview

“…Historically, LMICs have been the focus of substantial research capacity-building investments, ranging from the Rockefeller Foundation's International Clinical Epidemiology Network 22 to the U.S. National Institutes of Health Fogarty International Center’s (FIC) Global Infectious Diseases programs, Global Health Program for Fellows and Scholars, 23 and Medical Education Partnership Initiative. 24 Other recent efforts include the Wellcome Trust’s African Institutions Initiative 25 and Brazil’s Science without Borders, 26 in addition to activities supported by the U.S. Centers for Disease Control and Prevention, and the U.S. Agency for International Development. None of these investments, however, has specifically addressed the need for local mentoring models or approaches in LMICs.…”

External funding to strengthen capacity for research in low-income and middle-income countries: exigence, excellence and equity

“…First, we need to learn lessons from previous attempts at promoting excellence and equity, including the need for relevant evaluation indicators. For example, under the African Institutions Initiative funded by the Wellcome Trust, the consortia included links between universities with greater and lesser degrees of established research capacity, requiring indicators of evaluation other than those based on research quality 14…”