Use of programme theory to understand the differential effects of interventions across socio-economic groups in systematic reviews—a systematic methodology review

Abstract

BackgroundSystematic review guidance recommends the use of programme theory to inform considerations of if and how healthcare interventions may work differently across socio-economic status (SES) groups. This study aimed to address the lack of detail on how reviewers operationalise this in practice.MethodsA methodological systematic review was undertaken to assess if, how and the extent to which systematic reviewers operationalise the guidance on the use of programme theory in considerations of socio-economic inequalities in health. Multiple databases were searched from January 2013 to May 2016. Studies were included if they were systematic reviews assessing the effectiveness of an intervention and included data on SES. Two reviewers independently screened all studies, undertook quality assessment and extracted data. A narrative approach to synthesis was adopted.ResultsA total of 37 systematic reviews were included, 10 of which were explicit in the use of terminology for ‘programme theory’. Twenty-nine studies used programme theory to inform both their a priori assumptions and explain their review findings. Of these, 22 incorporated considerations of both what and how interventions do/do not work in SES groups to both predict and explain their review findings. Thirteen studies acknowledged 24 unique theoretical references to support their assumptions of what or how interventions may have different effects in SES groups. Most reviewers used supplementary evidence to support their considerations of differential effectiveness. The majority of authors outlined a programme theory in the “Introduction” and “Discussion” sections of the review to inform their assumptions or provide explanations of what or how interventions may result in differential effects within or across SES groups. About a third of reviews used programme theory to inform the review analysis and/or synthesis. Few authors used programme theory to inform their inclusion criteria, data extraction or quality assessment. Twenty-one studies tested their a priori programme theory.ConclusionsThe use of programme theory to inform considerations of if, what and how interventions lead to differential effects on health in different SES groups in the systematic review process is not yet widely adopted, is used implicitly, is often fragmented and is not implemented in a systematic way.Electronic supplementary materialThe online version of this article (10.1186/s13643-017-0638-9) contains supplementary material, which is available to authorized users.

What have we learned after ten years of systematic reviews in international development?

“…There have been a number of calls for the incorporation of programme theory into systematic reviews over the years (for example, Pawson 2002;Davies 2006;van der Knaap et al, 2008;Anderson et al 2011;Waddington et al 2012;Snilstveit 2012;Kneale, Thomas, and Harris 2015;Maden et al 2017;White 2018), as well as for multi-disciplinary working (for example, Thomas et al 2004;Snilstveit 2012;Oliver et al 2017;White 2018). Programme theory is usually incorporated into systematic reviews through logic models (flow diagrams which present the intervention causal chain from inputs through to final outcomes) or theories of change (which articulate the assumptions underlying the causal chain and the contexts and stakeholders for whom the intervention is relevant), and sometimes through economic, social or psychological theory to help articulate programme mechanisms.…”

Toward a theory-led metaframework for considering socioeconomic health inequalities within systematic reviews

“…The meta-framework is not designed to introduce rigidity into the review process (37). Its value lies in "its ability to allow an acceptable, systematic, tested and refined a posteriori reasoning rather than post hoc assumption of how interventions may work" (11,Discussion). The meta-framework is flexible enough to allow new factors and mechanisms to be incorporated and can be used, for example, to inform data extraction within a best-fit framework synthesis (18).…”

“…Systematic searches were undertaken in eight resources following guidance on searching for theory (18) (see table 2 and Appendix A). Theories were also identified opportunistically from within relevant theoretical papers, an earlier published work on the use of programme theory in SES focused systematic reviews (11) and informal discussions with health inequality experts. We excluded theories on the causes and determinants of inequalities since they do not focus on interventions.…”

“…Guidance on conducting systematic reviews that consider health inequalities encourages reviewers to develop an understanding, or 'programme theory'/logic model, from the outset of their review, of what works, for disadvantaged populations, under what circumstance (7)(8)(9)(10). However, much of the guidance assumes that reviewers can recognise a priori, what, how and why interventions may result in differential effects across different SES populations (11). Consequently, within the review guidance there is a lack of detail on the specific factors and mechanisms (i.e.…”

“…reviews (11). Thus, a framework that offers the potential to facilitate the identification of factors and mechanisms associated with what, how and why interventions may work across different SES groups, may help reviewers to operationalise the guidance on conducting systematic reviews that consider health inequalities.…”