Development and testing of an objective structured clinical exam (OSCE) to assess socio-cultural dimensions of patient safety competency

Abstract

BackgroundPatient safety (PS) receives limited attention in health professional curricula. We developed and pilot tested four Objective Structured Clinical Examination (OSCE) stations intended to reflect socio-cultural dimensions in the Canadian Patient Safety Institute's Safety Competency Framework.Setting and participants18 third year undergraduate medical and nursing students at a Canadian University.MethodsOSCE cases were developed by faculty with clinical and PS expertise with assistance from expert facilitators from the Medical Council of Canada. Stations reflect domains in the Safety Competency Framework (ie, managing safety risks, culture of safety, communication). Stations were assessed by two clinical faculty members. Inter-rater reliability was examined using weighted κ values. Additional aspects of reliability and OSCE performance are reported.ResultsAssessors exhibited excellent agreement (weighted κ scores ranged from 0.74 to 0.82 for the four OSCE stations). Learners’ scores varied across the four stations. Nursing students scored significantly lower (p<0.05) than medical students on three stations (nursing student mean scores=1.9, 1.9 and 2.7; medical student mean scores=2.8, 2.9 and 3.5 for stations 1, 2 and 3, respectively where 1=borderline unsatisfactory, 2=borderline satisfactory and 3=competence demonstrated). 7/18 students (39%) scored below ‘borderline satisfactory’ on one or more stations.ConclusionsResults show (1) four OSCE stations evaluating socio-cultural dimensions of PS achieved variation in scores and (2) performance on this OSCE can be evaluated with high reliability, suggesting a single assessor per station would be sufficient. Differences between nursing and medical student performance are interesting; however, it is unclear what factors explain these differences.

Undergraduate baccalaureate nursing students’ self-reported confidence in learning about patient safety in the classroom and clinical settings: An annual cross-sectional study (2010–2013)

“…The field would benefit from future research on related concepts such as health professional trainees patient safety knowledge and patient safety competence. Such endeavors would require other methods of inquiry that have not yet been widely explored in the literaturesee Ginsburg et al's (Ginsburg et al, 2014) pilot study of a patient safety OSCE (Objective Structured Clinical Exam) for a recent exception.…”

Non-technical skills assessments in undergraduate medical education: A focused BEME systematic review: BEME Guide No. 54

“…Nine papers were ultimately included in the qualitative and quantitative syntheses (Farnan et al, 2016;Jansson et al, 2015;Ginsburg et al, 2014;Daud-Gollotti et al, 2011;Madigosky et al, 2006;Aboumatar et al, 2012;Müller et al, 2012;Paxton and Rubinfeld, 2010;Thomas et al, 2015). The search flowchart is shown in Figure 1 and an overview of the included papers is shown in Table 1.…”

“…Four (Farnan et al, 2016;Jansson et al, 2015;Madigosky et al, 2006;Aboumatar et al, 2012) of the nine papers did not provide any descriptions of theoretical models or conceptual frameworks utilized for the non-technical skills assessments. A further three provided limited descriptions aligned with conceptual elements related to error wisdom and situational awareness (Ginsburg et al, 2014;Müller et al, 2012;Paxton and Rubinfeld, 2010). The remaining two gave significant detail of the frameworks used, one fully aligning with theoretical principles (Daud-Gallotti et al, 2011) of non-technical skills (Ginsburg et al, 2014) and the other identifying key theories related to error origins, describing the 'practice-effect' concept (Thomas et al, 2015).…”

The Effectiveness of a Course on Patient Safety Management on the Patient Safety Competencies by Final Year Bachelor of Nursing Students in the Netherlands

“…This framework underlines the sociocultural aspects of patient safety including culture (n=4), teamwork (n=6), communication (n=3), managing risk (n=3) and understanding human factors (n=3) [18] [19,20]. In addition, the OSCE includes awareness of the culture and the complexity of the healthcare system, as well as teamwork and the relevance of learning from skills [19,21].…”