The role of evaluation in iterative learning and implementation of quality of care interventions

Abstract

Background:The Lancet Global Health Commission (LGHC) has argued that quality of care (QoC) is an emergent property that requires an iterative process to learn and implement. Such iterations are required given that health systems are complex adaptive systems. Objective: This paper explores the multiple roles that evaluations need to play in order to help with iterative learning and implementation. We argue evaluation needs to shift from a summative focus toward an approach that promotes learning in complex systems. A framework is presented to help guide the iterative learning, and includes the dimensions of clinical care, person-centered care, continuum of care, and 'more than medicine. Multiple roles of evaluation corresponding to each of the dimensions are discussed. Methods: This paper is informed by reviews of the literature on QoC and the roles of evaluation in complex systems. The proposed framework synthesizes the multiple views of QoC. The recommendations of the roles of evaluation are informed both by review and experience in evaluating multiple QoC initiatives. Results: The specific roles of different evaluation approaches, including summative, realist, developmental, and participatory, are identified in relationship to the dimensions in our proposed framework. In order to achieve the potential of LGHC, there is a need to discuss how different evaluation approaches can be combined in a coherent way to promote iterative learning and implementation of QoC initiatives. Conclusion:One of the implications of the QoC framework discussed in the paper is that time needs to be spent upfront in recognizing areas in which knowledge of a specific intervention is not complete at the outset. This, of course, implies taking stock of areas of incompleteness in knowledge of context, theory of change, support structures needed in order for the program to succeed in specific settings. The role of evaluation should not be limited to only providing an external assessment, but an important goal in building evaluation capacity should be to promote adaptive management among planners and practitioners. Such iterative learning and adaptive management are needed to achieve the goals of sustainable development goals.

Reflections on Required Competencies For Health Systems Monitoring, Evaluation, and Learning In India

“…However, where a (funding) initiative seeks to operate across levels or functions of the health system, it will continue to present unique strategic and operational challenges. The pathway toward a learning health system requires ongoing iterative improvement in different parts of the system, operating at multiple different levels through developmental evaluation (Shah et al, 2021).…”

Building Spaces for Dialogues to Rethink Evaluator Competencies: Lessons from the Webinars Organized by the Evaluation Centre for Complex Health Interventions

“…How could evaluator competencies help us focus on both the multiple diverse contexts of individuals (Shah et al, 2021) and the need for a diversity of local solutions? How can evaluators be trained to throw better light on the heterogeneities of solutions that would be needed to address diverse needs across different contexts?…”

“…Yet, as we work on the complex problems of housing and health in different settings, we are struck by how we could do much more as a field to focus on the multiple dimensions of contexts and support structures in the planning, implementation, and sustainability of interventions. In a recent paper, we and our colleagues argued that iterative learning is critical in understanding what we termed the five I's of contexts: "Infrastructural, Institutional, Interpersonal, Individual, and Intersectional" (Pawson et al, 2004;Shah et al, 2021). Some questions for reflection that emerged in multiple webinars include: Has our practice been sensitive to contexts?…”

Implementing value-based healthcare using a digital health exchange platform to improve pregnancy and childbirth outcomes in urban and rural Kenya

“…Over the past decade, investments in improving maternal, neonatal and child health (MNCH) in low- and middle-income countries (LMICs) have largely focused on reducing financial and geographical barriers to access skilled delivery and incentivizing adherence to antenatal guidelines improving scale, scope and quality of healthcare delivery and stimulating utilization by reducing financial barriers ( 1 , 2 ). Despite these investments, maternal mortality rates (MMR) and neonatal mortality rates (NMR) are still high in LMICs.…”

“…This raises the question what more is required to meet patients' needs and to achieve the targets of SDG 3? Research shows that improving accessibility and adherence alone does not guarantee better outcomes and that a broader set of interventions aimed at quality of care is needed ( 2 , 5 , 6 ).…”