Can cultural competency self-assessment predict OSCE performance?

Abstract

resulting in a pass rate of 51%. Scores ranged from 34.8% to 70.5%. Overall, 80% of students considered the OSCE to be a more objective clinical assessment tool, most appreciated the time-efficiency of the process, and all respondents were in favour of adopting the OSCE as a permanent tool. Future plans in the Department of Internal Medicine include the adoption of the OSCE in Years 4 and 6. In addition, the Departments of Psychiatry and Preventive Dentistry are investigating possible uses of the OSCE model. Can cultural competency self-assessment predict OSCE performance?Ming-Jung Ho, Keng-Lin Lee & Alexander R Green Context and setting Although cultural competency training is required by various accreditation bodies in medical education, there is no agreement upon how best to evaluate such training programmes. According to systematic reviews, most studies evaluating cultural competency training of health professionals utilise self-assessments. Few studies have employed objective structured clinical examinations (OSCEs). No study has reported the relationship between self-assessments and OSCEs in evaluating cultural competency training. Why the idea was necessary The OSCE is generally regarded as a better method of measuring competency in clinical skills than self-assessment. However, conducting an OSCE is so much more demanding than administrating a self-efficacy survey that the latter is often chosen as the means to measure cultural competency skills. This study addresses the unanswered question of whether self-assessment can substitute for the OSCE in measuring cross-cultural communication skills. What was done We recruited 57 Year 5 students at our medical school to participate in the study between January 2006 and June 2006. Students filled out a survey containing the Inventory for Assessing the Process of Cultural Competence among Healthcare Professionals-Revised (IAPCC-R), which has 5 subscales, the California Brief Multicultural Competence Scale (CBMCS), which has 4 subscales, and a survey designed to measure US residentsÕ preparedness to deliver cross-cultural care (CCC), which has 2 subscales. The CronbachÕs alpha coefficients for the 3 scales were 0.71, 0.84 and 0.92, respectively. All students were then evaluated with an objective structured clinical examination (OSCE) which tested their ability to explore sociocultural factors influencing a standardised patient adherence to chronic disease treatment. Multiple regressions were conducted to predict the patient perspectives (PP) and social factors (SF) subscales of the OSCE. All 11 subscales of the IAPCC-R, CBMCS and CCC were selected with the stepwise method. We analysed the data using SAS Version 9.1. Evaluation of results and impact In the PP regression model, PP was predicted by the ÔskillÕ subscale of the IAPCC-R and the ÔawarenessÕ subscale of the CBMCS. Overall model test was significant (F[2,56] = 4.59, P = 0.0145, R 2 = 0.15) and parameter estimates were significant at a level of 0.05. The estimated model was PP = 1.32 -0.18 · skill + 0.19 · ...

Clinical teachers' perspectives on cultural competence in medical education

“…However, they also found the benefit seemed to degrade over time in a follow‐up study. In addition, a discrepancy was found between the students' self‐assessment of competence and their observed competence as demonstrated by OSCE scores . Skelton cast doubts on OSCEs as a tool, cautioning that it can descend into formulaic, checklist learning.…”

“…In addition, a discrepancy was found between the students' self-assessment of competence and their observed competence as demonstrated by OSCE scores. 31 Skelton 32 cast doubts on OSCEs as a tool, cautioning that it can descend into formulaic, checklist learning. A concern was also raised over what 'intercultural competence' versus good interpersonal interviewing skills in OSCEs were termed.…”

Self-Assessed Competency at Working with a Medical Interpreter Is Not Associated with Knowledge of Good Practice

“…Despite their well-known weaknesses, self-assessments remain a practical evaluation method for large-scale studies. Integrating vignettes such as the one we have tested into surveys may be one way to strengthen self-assessments [32] . Nonetheless, further work is needed to develop and test vignettes which accurately assess respondents' knowledge of good practice.…”

Self-assessment of intercultural communication skills: a survey of physicians and medical students in Geneva, Switzerland

“…Unfortunately, while many of these methods have been used successfully to evaluate intercultural communication skills in small-scale studies and training contexts [31-35], they are too costly and impractical for larger-scale assessments such as described in this paper. Skills self-assessment is likely to continue to be a widely-used method for identifying training needs [36], and future efforts should focus on developing more objective measures of cultural competency that can be integrated into survey instruments [37]. …”