How hard can it be to include research evidence and evaluation in local health policy implementation? Results from a mixed methods study

Abstract

BackgroundAlthough an evidence-based approach is the ideal model for planning and delivering healthcare, barriers exist to using research evidence to implement and evaluate service change. This paper aims to inform policy implementation and evaluation by understanding the role of research evidence at the local level through implementation of a national chronic conditions management policy.MethodsWe conducted a national email survey of health service commissioners at the most devolved level of decision-making in Wales (Local Health Boards – LHBs) followed by in-depth interviews with representatives of LHBs, purposively selecting five to reflect geographic and economic characteristics. Survey data were analysed descriptively; we used thematic analysis for interview data.ResultsAll LHBs (n = 22) completed questionnaires. All reported they routinely assessed the research literature before implementing interventions, but free-text answers revealed wide variation in approach. Most commonly reported information sources included personal contacts, needs assessments, information or research databases. No consistent approach to evaluation was reported. Frequently reported challenges were: insufficient staff capacity (17/22); limited skills, cost, limited time, competing priorities (16/22); availability and quality of routine data (15/22). Respondents reported they would value central guidance on evaluation.Five interviews were held with managers from the five LHBs contacted. Service delivery decisions were informed by Welsh Government initiatives and priorities, budgets, perceived good practice, personal knowledge, and local needs, but did not include formal research evidence, they reported. Decision making was a collaborative process including clinical staff, patient representatives, and partner organization managers with varying levels of research experience. Robust evaluation data were required, but they were constrained by a lack of skills, time, and resources. They viewed evaluation as a means of demonstrating that targets had been met.ConclusionsThere is a gap between evidence-based aims of national health policy and how health services are commissioned, implemented, and evaluated at local level. Commissioners and managers are unable to routinely incorporate research evidence. If health services research is to identify most effective ways to implement high quality care, it should be incorporated into commissioning and service delivery. Local commissioners and managers need to build the critical use of research evidence and evaluation into health policy implementation at local level in order to provide consistent and effective healthcare services.

Sustainability in Health care by Allocating Resources Effectively (SHARE) 5: developing a model for evidence-driven resource allocation in a local healthcare setting

“…Monash Health is not unique and these issues are commonplace in health services around the world [8, 40, 44, 51, 57–60]. This indicates enormous opportunities for improvement through strategies that address these limitations.…”

The SPIRIT Action Framework: A structured approach to selecting and testing strategies to increase the use of research in policy

“…Although the Framework drew on literature available at its time of development (2012), subsequent work (e.g. Brown, 2012;Ettelt et al, 2013;Evans et al, 2013;Moat et al, 2013;Oliver et al, 2014;Panisset et al, 2012;Pitchforth et al, 2013;Yost et al, 2014) have largely confirmed previous reviews and the basis of the Framework.…”

Sustainability in Health care by Allocating Resources Effectively (SHARE) 6: investigating methods to identify, prioritise, implement and evaluate disinvestment projects in a local healthcare setting

“…However, there is a lack of information to guide local healthcare services regarding an organisational approach to disinvestment [3–12]. …”