BackgroundAlthough an evidence-based approach is the ideal model for planning and delivering healthcare, barriers exist to using research evidence to implement and evaluate service change. This paper aims to inform policy implementation and evaluation by understanding the role of research evidence at the local level through implementation of a national chronic conditions management policy.MethodsWe conducted a national email survey of health service commissioners at the most devolved level of decision-making in Wales (Local Health Boards – LHBs) followed by in-depth interviews with representatives of LHBs, purposively selecting five to reflect geographic and economic characteristics. Survey data were analysed descriptively; we used thematic analysis for interview data.ResultsAll LHBs (n = 22) completed questionnaires. All reported they routinely assessed the research literature before implementing interventions, but free-text answers revealed wide variation in approach. Most commonly reported information sources included personal contacts, needs assessments, information or research databases. No consistent approach to evaluation was reported. Frequently reported challenges were: insufficient staff capacity (17/22); limited skills, cost, limited time, competing priorities (16/22); availability and quality of routine data (15/22). Respondents reported they would value central guidance on evaluation.Five interviews were held with managers from the five LHBs contacted. Service delivery decisions were informed by Welsh Government initiatives and priorities, budgets, perceived good practice, personal knowledge, and local needs, but did not include formal research evidence, they reported. Decision making was a collaborative process including clinical staff, patient representatives, and partner organization managers with varying levels of research experience. Robust evaluation data were required, but they were constrained by a lack of skills, time, and resources. They viewed evaluation as a means of demonstrating that targets had been met.ConclusionsThere is a gap between evidence-based aims of national health policy and how health services are commissioned, implemented, and evaluated at local level. Commissioners and managers are unable to routinely incorporate research evidence. If health services research is to identify most effective ways to implement high quality care, it should be incorporated into commissioning and service delivery. Local commissioners and managers need to build the critical use of research evidence and evaluation into health policy implementation at local level in order to provide consistent and effective healthcare services.
AimWe evaluated the introduction of a predictive risk stratification model (PRISM) into primary care. Contemporaneously National Health Service (NHS) Wales introduced Quality and Outcomes Framework payments to general practices to focus care on those at highest risk of emergency admission to hospital. The aim of this study was to evaluate the costs and effects of introducing PRISM into primary care.MethodsRandomised stepped wedge trial with 32 general practices in one Welsh health board. The intervention comprised: PRISM software; practice-based training; clinical support through two ‘general practitioner (GP) champions’ and technical support. The primary outcome was emergency hospital admissions.ResultsAcross 230 099 participants, PRISM implementation increased use of health services: emergency hospital admission rates by 1 % when untransformed (while change in log-transformed rate ΔL=0.011, 95% CI 0.010 to 0.013); emergency department (ED) attendance rates by untransformed 3 % (while ΔL=0.030, 95% CI 0.028 to 0.032); outpatient visit rates by untransformed 5 % (while ΔL=0.055, 95% CI 0.051 to 0.058); the proportion of days with recorded GP activity by untransformed 1 % (while ΔL=0.011, 95% CI 0.007 to 0.014) and time in hospital by untransformed 3 % (while ΔL=0.029, 95% CI 0.026 to 0.031). Thus NHS costs per participant increased by £76 (95% CI £46 to £106).ConclusionsIntroduction of PRISM resulted in a statistically significant increase in emergency hospital admissions and use of other NHS services without evidence of benefits to patients or the NHS.
BackgroundWales faces serious public health challenges, with relatively low life expectancies and wide inequalities in life expectancy with associated pressures on the National Health Service (NHS) at a time of financial recession. This has led to growing recognition of the need to better understand the range of health improvement and prevention programmes across Welsh Government, NHS, local government and voluntary sector agencies.MethodsThe Minister for Health and Social Care commissioned Public Health Wales, the single national public health organisation, to establish a Health Improvement Advisory Group, to oversee a Programme Budgeting and Marginal Analysis (PBMA) expert panel. The panel drew on evidence from a range of sources to explore potential alternative modes of health improvement initiative delivery across Wales. Electronic voting was used to agree an appropriate time horizon for health improvement programme outcomes, main objective of the health improvement review and criteria for evaluating candidate services for disinvestment and investment. The panel also used electronic voting to state whether they wished to disinvest or invest in a candidate service.ResultsThe review identified a budget of £15.1 million, spanning 10 Welsh Government priority areas, and 6 life course stages. Due to lack of evidence the panel recommended total disinvestment in 7 out of 25 initiatives releasing £1.5 million of resources, and partial disinvestment in a further 3 interventions releasing £7.3 million of resources. The panel did not recommend increasing investment in any of the 25 initiatives under review. Marginal analyses prioritised child health, mental health and wellbeing and tobacco control as key areas for investment.ConclusionsWales is championing a concept of “prudent healthcare”. The PBMA exercise undertaken here was a transparent evidence-based tool to reach decisions about potential for disinvestment and reinvestment in health improvement strategies. It also demonstrates the potential wider application at a national level across government public health functions, to ensure resources are most cost-effectively deployed, with due consideration for equity.Electronic supplementary materialThe online version of this article (doi:10.1186/1471-2458-14-837) contains supplementary material, which is available to authorized users.
Background With a higher proportion of older people in the UK population, new approaches are needed to reduce emergency hospital admissions, thereby shifting care delivery out of hospital when possible and safe. Study aim To evaluate the introduction of predictive risk stratification in primary care. Objectives To (1) measure the effects on service usage, particularly emergency admissions to hospital; (2) assess the effects of the Predictive RIsk Stratification Model (PRISM) on quality of life and satisfaction; (3) assess the technical performance of PRISM; (4) estimate the costs of PRISM implementation and its effects; and (5) describe the processes of change associated with PRISM. Design Randomised stepped-wedge trial with economic and qualitative components. Setting Abertawe Bro Morgannwg University Health Board, south Wales. Participants Patients registered with 32 participating general practices. Intervention PRISM software, which stratifies patients into four (emergency admission) risk groups; practice-based training; and clinical support. Main outcome measures Primary outcome – emergency hospital admissions. Secondary outcomes – emergency department (ED) and outpatient attendances, general practitioner (GP) activity, time in hospital, quality of life, satisfaction and costs. Data sources Routine anonymised linked health service use data, self-completed questionnaires and staff focus groups and interviews. Results Across 230,099 participants, PRISM implementation led to increased emergency admissions to hospital [ΔL = 0.011, 95% confidence interval (CI) 0.010 to 0.013], ED attendances (ΔL = 0.030, 95% CI 0.028 to 0.032), GP event-days (ΔL = 0.011, 95% CI 0.007 to 0.014), outpatient visits (ΔL = 0.055, 95% CI 0.051 to 0.058) and time spent in hospital (ΔL = 0.029, 95% CI 0.026 to 0.031). Quality-of-life scores related to mental health were similar between phases (Δ = –0.720, 95% CI –1.469 to 0.030); physical health scores improved in the intervention phase (Δ = 1.465, 95% CI 0.774 to 2.157); and satisfaction levels were lower (Δ = –0.074, 95% CI – 0.133 to –0.015). PRISM implementation cost £0.12 per patient per year and costs of health-care use per patient were higher in the intervention phase (Δ = £76, 95% CI £46 to £106). There was no evidence of any significant difference in deaths between phases (9.58 per 1000 patients per year in the control phase and 9.25 per 1000 patients per year in the intervention phase). PRISM showed good general technical performance, comparable with existing risk prediction tools (c-statistic of 0.749). Qualitative data showed low use by GPs and practice staff, although they all reported using PRISM to generate lists of patients to target for prioritised care to meet Quality and Outcomes Framework (QOF) targets. Limitations In Wales during the study period, QOF targets were introduced into general practice to encourage targeting care to those at highest risk of emergency admission to hospital. Within this dynamic context, we therefore evaluated the combined effects of PRISM and this contemporaneous policy initiative. Conclusions Introduction of PRISM increased emergency episodes, hospitalisation and costs across, and within, risk levels without clear evidence of benefits to patients. Future research (1) Evaluation of targeting of different services to different levels of risk; (2) investigation of effects on vulnerable populations and health inequalities; (3) secondary analysis of the Predictive Risk Stratification: A Trial in Chronic Conditions Management data set by health condition type; and (4) acceptability of predictive risk stratification to patients and practitioners. Trial and study registration Current Controlled Trials ISRCTN55538212 and PROSPERO CRD42015016874. Funding The National Institute for Health Research Health Services Delivery and Research programme.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.