Evaluating the Impact of the Medical Education Partnership Initiative at the University of Zimbabwe College of Health Sciences Using the Most Significant Change Technique

Abstract

Problem In medical education, evaluating outcomes from programs intended to transform attitudes or influence career trajectories using conventional methods of monitoring is often difficult. To address this problem, the authors adapted the most significant change (MSC) technique to gain a more comprehensive understanding of the impact of the Medical Education Partnership Initiative (MEPI) program at the University of Zimbabwe College of Health Sciences. Approach In 2014–2015, the authors applied the MSC to systematically examine the personal significance and level of positive transformation individuals attributed to their MEPI participation. Interviews were conducted with 28 participants nominated by MEPI program leaders. The authors coded results inductively for prevalent themes in participants’ stories and prepared profiles with representative quotes to place the stories in context. Stakeholders selected 9 themes and 18 stories to illustrate the most significant changes. Outcomes Six themes (or outcomes) were expected, as they aligned with MEPI goals—becoming a better teacher, becoming a better clinician, increased interest in teaching, increased interest in research, new career pathways (including commitment to practice in Zimbabwe), and improved research skills. However, three themes were unexpected—increased confidence, expanded interprofessional networks, and improved interpersonal interactions. Next Steps We found the MSC to be a useful and systematic evaluation approach for large, complex, and transformative initiatives like MEPI. The MSC seemed to encourage participant reflection, support values inquiry by program leaders, and provide insights into the personal and cultural impacts of MEPI. Additional trial applications of the MSC technique in academic medicine are warranted.

Evaluating a peer-led wellbeing programme for doctors-in-training during the COVID-19 pandemic in Victoria, Australia, using the Most Significant Change technique

“…The centrality of storytelling as the data gathering method makes MSC well suited for evaluating healthcare programmes across various cultural contexts, particularly those targeting change at a personal level ( Tonkin et al, 2021 ). Examples include promotion of childbirth spacing in Nigeria ( Aisiri et al, 2020 ), maternal community health worker training in Indonesia ( Limato et al, 2018 ), a medical education initiative to increase healthcare worker retention in sub-Saharan Africa ( Connors et al, 2017 ), and cultural safety training for Colombian medical students ( Pimental et al, 2021 ).…”

“…Reasons given for choosing the MSC technique in these evaluations include its participatory nature, the focus on participant experience, provision of insight into how change occurred, and ability to capture unintended outcomes. A key strength cited is the potential for storytelling to contribute to empowerment and further the change process ( Aisiri et al, 2020 ; Connors et al, 2017 ; Limato et al, 2017 ; Tonkin et al., 2021 ).…”

Benefits of transport subsidisation: Comparing findings from a customer perception survey and Most Significant Change Technique interviews

“…The method concentrates on anecdotal evidence of change that is often missed by conventional quantitative monitoring techniques (Wilder & Walpole 2008). Most Significant Change adds value to measuring qualitative change indicators that are difficult to measure (Connors et al 2017;Kloosterman, Benning & Fyles 2012). Wilson (2014) considered MSC well-suited for public sector programmes that are continuous in nature, where outcomes may vary significantly between beneficiaries or where there may not be prior agreement between stakeholders on which outcomes are most important to the programme.…”

Remotely and collaboratively evaluating a campus-based therapy dog program during the COVID-19 pandemic

“…The second column in Table 1 briefly outlines the steps involved in the MSC process. While numerous published evaluations have used this technique (e.g., Aisiri et al, 2020 ; Limato et al, 2018 ), and modified versions of it (e.g., Connors et al, 2017 ; Ho et al, 2015 ), our time-restricted, remote approach required further adaptations.…”