Realist evaluation of a complex integrated care programme: protocol for a mixed methods study

Abstract

IntroductionThe lack of understanding of how complex integrated care programmes achieve their outcomes due to the lack of acceptable methods leads to difficulties in the development, implementation, adaptation and scaling up of similar interventions. In this study, we evaluate an integrated care network, the National University Health System (NUHS) Regional Health System (RHS), consisting of acute hospitals, step down care, primary care providers, social services and community partners using a theory-driven realist evaluation approach. This study aims to examine how and for whom the NUHS-RHS works to improve healthcare utilisations, outcomes, care experiences and reduce healthcare costs. By using a realist approach that balances the needs of context-specific evaluation with international comparability, this study carries the potential to address current research gaps.Methods and analysisThis evaluation will be conducted in three research phases: (1) development of initial programme theory (IPT) underlying the NUHS-RHS; (2) testing of programme theory using empirical data; and (3) refinement of IPT. IPT was elicited and developed through reviews of programme documents, informal discussions and in-depth interviews with relevant stakeholders. Then, a convergent parallel mixed method study will be conducted to assess context (C), mechanisms (M) and outcomes (O) to test the IPT. Findings will then be analysed according to the realist evaluation formula of CMO in which findings on the context, mechanisms will be used to explain the outcomes. Finally, based on findings gathered, IPT will be refined to highlight how to improve the NUHS-RHS by detailing what works (outcome), as well as how (mechanisms) and under what conditions (context).Ethics and disseminationThe National Healthcare Group, Singapore, Domain Specific Review Board reviewed and approved this study protocol. Study results will be published in international peer-reviewed journals and presented at conferences and internally to NUHS-RHS and Ministry of Health, Singapore.

Challenges in turning a great idea into great health policy: the case of integrated care

“…In cases of social (health) reform, determining the control group is difficult, and policy research may encounter threats to both internal and external validity [43]. Many other methodologies have been developed and applied in health care policy experimentation [44], such as realist evaluation [45] and contribution analysis [46]. Discussions in these domains are still underway as these methodologies each have their own strengths and challenges.…”

Conducting a Cost-Benefit Analysis of Transitional Care Programmes: The Key Challenges and Recommendations

“…In our evaluation study [6], the number of lives saved was found to have improved [7]. Cost-utility analysis measures more aspects of health and well-being but was not appropriate without findings demonstrating improvements in health-related quality of life.…”

Mixed methods protocol for a realist evaluation of electronic personal health records design features and use to support medication adherence (ePHRma)

“…A realist evaluation includes a contextual evaluation (in what circumstances),37 a process evaluation (what works for whom)38 and outcomes evaluation (how). This holistic evaluation approach makes realist evaluation framework an ideal candidate to deal with the uncertainty and the complexity of interventions in medical informatics 39 40…”