From summative to developmental

Abstract

Purpose Models of integrated care are prime examples of complex interventions, incorporating multiple interacting components that work through varying mechanisms to impact numerous outcomes. The purpose of this paper is to explore summative, process and developmental approaches to evaluating complex interventions to determine how to best test this mess. Design/methodology/approach This viewpoint draws on the evaluation and complex intervention literatures to describe the advantages and disadvantages of different methods. The evaluation of the electronic patient reported outcomes (ePRO) mobile application and portal system is presented as an example of how to evaluate complex interventions with critical lessons learned from this ongoing study. Findings Although favored in the literature, summative and process evaluations rest on two problematic assumptions: it is possible to clearly identify stable mechanisms of action; and intervention fidelity can be maximized in order to control for contextual influences. Complex interventions continually adapt to local contexts, making stability and fidelity unlikely. Developmental evaluation, which is more conceptually aligned with service-design thinking, moves beyond these assumptions, emphasizing supportive adaptation to ensure meaningful adoption. Research limitations/implications Blended approaches that incorporate service-design thinking and rely more heavily on developmental strategies are essential for complex interventions. To maximize the benefit of this approach, three guiding principles are suggested: stress pragmatism over stringency; adopt an implementation lens; and use multi-disciplinary teams to run studies. Originality/value This viewpoint offers novel thinking on the debate around appropriate evaluation methodologies to be applied to complex interventions like models of integrated care.

Technology-Enabled Collaborative Care for Type-2 Diabetes and Mental Health (TECC-D): Findings From a Mixed Methods Feasibility Trial of a Responsive Co-Designed Virtual Health Coaching Intervention

“…Integrated care models are complex interventions that seek to integrate services within and across organizational and professional boundaries. These models themselves are complex not only because of their use of governance, systems, and technology to achieve these aims, but because of the often complex nature of the individuals they serve [19]. Reflective of a deliberate shift toward people-driven care [20], rather than merely peoplecentred care, co-design centers the expressed needs of those living with the condition in the design process.…”

Exploring the impact of evaluation on learning and health innovation sustainability: protocol for a realist synthesis

“…LHS are complex social systems in which the concepts of dynamism and adaptation of the system to an ever-changing context are embodied [ 1 ]. Thus, investing effort in the search for standardized evaluation mechanisms that produce predefined effects through “rational” behaviors may be utopian, if not futile, and attention should rather be focused on how evaluation enables the development of dynamic capacities for the continuous adaptation and improvement of health systems through knowledge flows [ 18 ].…”