Developing a framework for evaluating the impact of Healthcare Improvement Science Education across Europe: a qualitative study

Abstract

PurposeFrontline healthcare professionals are well positioned to improve the systems in which they work. Educational curricula, however, have not always equipped healthcare professionals with the skills or knowledge to implement and evaluate improvements. It is important to have a robust and standardized framework in order to evaluate the impact of such education in terms of improvement, both within and across European countries. The results of such evaluations will enhance the further development and delivery of healthcare improvement science (HIS) education. We aimed to describe the development and piloting of a framework for prospectively evaluating the impact of HIS education and learning.MethodsThe evaluation framework was designed collaboratively and piloted in 7 European countries following a qualitative methodology. The present study used mixed methods to gather data from students and educators. The framework took the Kirkpatrick model of evaluation as a theoretical reference.ResultsThe framework was found to be feasible and acceptable for use across differing European higher education contexts according to the pilot study and the participants’ consensus. It can be used effectively to evaluate and develop HIS education across European higher education institutions.ConclusionWe offer a new evaluation framework to capture the impact of HIS education. The implementation of this tool has the potential to facilitate the continuous development of HIS education.

Mapping the Status of Healthcare Improvement Science through a Narrative Review in Six European Countries

“…The patient safety approach is mainly how HIS is understood and practiced within the Spanish context, though some references were found regarding the practice of healthcare improvement in cooperation projects developed in low-income countries by Spanish authors [55,56]. Furthermore, efforts have been made in the Spanish context to create an evaluation framework for HIS training in practice [5].…”

“…In 2013, the project, funded by the European Commission and composed of seven countries working in partnership, aimed to develop accredited evidence-based improvement science education for healthcare professionals in Europe through an all-partner consensus [4]. The overall activities were a literature review about the topic, a scoping exercise of professional competencies and capabilities required to practice HIS, an educational gap analysis, a specific HIS evaluation framework, and the development of a consensus definition of HIS [3,5], also to create four accredited, interdisciplinary learning modules based on experiential HIS learning pedagogies [6]. Working with a multidisciplinary team of healthcare workers, educators and health services researchers from Italy, Poland, Romania, Slovenia, Spain, England, and Scotland, the project aimed to develop a collective understanding of HIS and the knowledge, tools, and techniques required to extend it.…”

“…Such is the case of the International Summer Programme on HIS, carried out by the University of Alicante (Spain) every year since 2016, in which the ISTEW modules’ contents have been applied [7]. Furthermore, the HIS evaluation framework created in the ISTEW has been piloted in more European and non-European contexts [5].…”

The United Kingdom Field Epidemiology Training Programme: meeting programme objectives

“…The study framework, Kirkpatrick’s model for evaluating training, was principally designed for industry, but is often used to frame evaluations of health professional training [19-21]. FETP evaluations do not often report on behaviour or results (levels 3 and 4 of the Kirkpatrick model), which is consistent with other areas of health training, where organisational, stakeholder and population benefits are less frequently assessed than impacts on learners [26,27].…”

“…The framework for the evaluation was Kirkpatrick’s model for evaluating training [19-21]. The study was focused on levels 3 and 4 of this model.…”