Challenges, solutions and future directions in the evaluation of service innovations in health care and public health

Raine, Rosalind; Fitzpatrick, Ray; Barratt, Helen; Bevan, Gywn; Black, Nick; Boaden, Ruth; Bower, Peter; Campbell, Marion K; Denis, Jean‐Louis; Devers, Kelly J.; Dixon‐Woods, Mary; Fallowfield, Lesley; Forder, Julien E.; Foy, Robbie; Freemantle, Nick; Fulop, Naomi; Gibbons, Elizabeth; Gillies, Clare; Goulding, Lucy; Grieve, Richard; Grimshaw, Jeremy; Howarth, Emma; Lilford, Richard; McDonald, Ruth; Moore, Graham; Moore, Laurence; Newhouse, Robin; O’Cathain, Alicia; Or, Zeynep; Papoutsi, Chrysanthi; Prady, Stephanie L.; Rycroft‐Malone, Jo; Sekhon, Jasjeet S.; Turner, Simon; Watson, Samuel; Zwarenstein, Merrick

doi:10.3310/hsdr04160

Cited by 126 publications

(135 citation statements)

References 328 publications

Supporting

Mentioning

126

Contrasting

Unclassified

Order By: Relevance

“…The core configuration of assessed study areas differed notably among the eight countries. In this regard, the availability of information on the health system at local and national levels is essential for providing informed evidence for health care planning (Raine et al 2016).…”

Section: Discussionmentioning

confidence: 99%

Standard comparison of local mental health care systems in eight European countries

Gutiérrez-Colosía

Salvador‐Carulla

Salinas-Pérez

et al. 2017

Epidemiol Psychiatr Sci

View full text Add to dashboard Cite

Aims. There is a need of more quantitative standardised data to compare local Mental Health Systems (MHSs) across international jurisdictions. Problems related to terminological variability and commensurability in the evaluation of services hamper like-with-like comparisons and hinder the development of work in this area. This study was aimed to provide standard assessment and comparison of MHS in selected local areas in Europe, contributing to a better understanding of MHS and related allocation of resources at local level and to lessen the scarcity in standard service comparison in Europe. This study is part of the Seventh Framework programme REFINEMENT (Research on Financing Systems' Effect on the Quality of Mental Health Care in Europe) project.Methods. A total of eight study areas from European countries with different systems of care (Austria, England, Finland, France, Italy, Norway, Romania, Spain) were analysed using a standard open-access classification system (Description and Evaluation of Services for Long Term Care in Europe, DESDE-LTC). All publicly funded services universally accessible to adults (≥18 years) with a psychiatric disorder were coded. Care availability, diversity and capacity were compared across these eight local MHS.Results. The comparison of MHS revealed more community-oriented delivery systems in the areas of England (Hampshire) and Southern European countries (Verona -Italy and Girona -Spain). Community-oriented systems with a higher proportion of hospital care were identified in Austria (Industrieviertel) and Scandinavian countries (Sør-Trøndelag in Norway and Helsinki-Uusimaa in Finland), while Loiret (France) was considered as a predominantly hospital-based system. The MHS in Suceava (Romania) was still in transition to community care.Conclusions. There is a significant variation in care availability and capacity across MHS of local areas in Europe. This information is relevant for understanding the process of implementation of community-oriented mental health care in local areas. Standard comparison of care provision in local areas is important for context analysis and policy planning.

show abstract

Section: Discussionmentioning

confidence: 99%

Standard comparison of local mental health care systems in eight European countries

Gutiérrez-Colosía

Salvador‐Carulla

Salinas-Pérez

et al. 2017

Epidemiol Psychiatr Sci

View full text Add to dashboard Cite

show abstract

“…85 Randomising organisations (or departments within organisations) and then delivering new interventions within the organisations might overcome some of the problems identified. For example, once an organisation has been allocated to intervention, all employees could potentially be allocated to a new model of case management, while control clusters could continue with care as usual.…”

Section: Alternative Recruitment Methodsmentioning

confidence: 99%

Collaborative case management to aid return to work after long-term sickness absence: a pilot randomised controlled trial

et al. 2018

Self Cite

View full text Add to dashboard Cite

BackgroundDespite high levels of employment among working-age adults in the UK, there is still a significant minority who are off work with ill health at any one time (so-called ‘sickness absence’). Long-term sickness absence results in significant costs to the individual, to the employer and to wider society.ObjectiveThe overall objective of the intervention was to improve employee well-being with a view to aiding return to work. To meet this aim, a collaborative case management intervention was adapted to the needs of UK employees who were entering or experiencing long-term sickness absence.DesignA pilot randomised controlled trial, using permuted block randomisation. Recruitment of patients with long-term conditions in settings such as primary care was achieved by screening of routine records, followed by mass mailing of invitations to participants. However, the proportion of patients responding to such invitations can be low, raising concerns about external validity. Recruitment in the Case Management to Enhance Occupational Support (CAMEOS) study used this method to test whether or not it would transfer to a population with long-term sickness absence in the context of occupational health (OH).ParticipantsEmployed people on long-term sickness absence (between 4 weeks and 12 months). The pilot was run with two different collaborators: a large organisation that provided OH services for a number of clients and a non-profit community-based organisation.InterventionCollaborative case management was delivered by specially trained case managers from the host organisations. Sessions were delivered by telephone and supported use of a self-help handbook. The comparator was usual care as provided by participants’ general practitioner (GP) or OH provider. This varied for participants according to the services available to them. Neither participants nor the research team were blind to randomisation.Main outcome measuresRecruitment rates, intervention delivery and acceptability to participants were the main outcomes. Well-being, as measured by the Clinical Outcomes in Routine Evaluation Outcome Measure (CORE-OM), and return-to-work rates were also recorded.ResultsIn total, over 1000 potentially eligible participants were identified across the sites and invited to participate. However, responses were received from just 61 of those invited (5.5%), of whom 16 (1.5%) were randomised to the trial (seven to treatment, nine to control). Detailed information on recruitment methods, intervention delivery, engagement and acceptability is presented. No harms were reported in either group.ConclusionsThis pilot study faced a number of barriers, particularly in terms of recruitment of employers to host the research. Our ability to respond to these challenges faced several barriers related to the OH context and the study set up. The intervention seemed feasible and acceptable when delivered, although caution is required because of the small number of randomised participants. However, employees’ lack of engagement in the research might imply that they did not see the intervention as valuable.Future workDeveloping effective and acceptable ways of reducing sickness absence remains a high priority. We discuss possible ways of overcoming these challenges in the future, including incentives for employers, alternative study designs and further modifications to recruitment methods.Trial registrationCurrent Controlled Trials ISRCTN33560198.FundingThis project was funded by the NIHR Public Health Research programme and will be published in full inPublic Health Research; Vol. 6, No. 2. See the NIHR Journals Library website for further project information.

show abstract

“…As argued by Raines et al, (2016) 'the value of shifting from the traditionally used binary question of effectiveness, towards a more sophisticated exploration' is warranted, understanding the 'characterisation of interventions and their contexts of implementation' . 21 As highlighted later in the same report, knowledge translation is not a passive process. Many clinicians do not always engage with evidence-based practice and the effectiveness of interventions varies across different contexts.…”

Section: Metaanalysesmentioning

confidence: 85%

“…These have a particular focus on 'what works, for whom, why and in what circumstances' . 52 Again, such an approach can help address many of the stages in Glasziou and Haynes's model. Health service interventions commonly consist of a number of components that can act both independently and inter-dependently.…”

Section: Process Evaluationsmentioning

confidence: 99%

Implementation and trial evidence: a plea for fore-thought

et al. 2017

View full text Add to dashboard Cite

show abstract

Challenges, solutions and future directions in the evaluation of service innovations in health care and public health

Abstract: HeadlineEvaluating service innovations in health care and public health requires flexibility, collaboration and pragmatism; this collection identifies robust, innovative and mixed methods to inform such evaluations.

Cited by 126 publications

References 328 publications

Standard comparison of local mental health care systems in eight European countries

Standard comparison of local mental health care systems in eight European countries

Collaborative case management to aid return to work after long-term sickness absence: a pilot randomised controlled trial

Implementation and trial evidence: a plea for fore-thought

Contact Info

Product

Resources

About