Self-perceived competence correlates poorly with objectively measured competence in Evidence Based Medicine among medical students

Abstract

BackgroundPrevious studies report various degrees of agreement between self-perceived competence and objectively measured competence in medical students. There is still a paucity of evidence on how the two correlate in the field of Evidence Based Medicine (EBM). We undertook a cross-sectional study to evaluate the self-perceived competence in EBM of senior medical students in Malaysia, and assessed its correlation to their objectively measured competence in EBM.MethodsWe recruited a group of medical students in their final six months of training between March and August 2006. The students were receiving a clinically-integrated EBM training program within their curriculum. We evaluated the students' self-perceived competence in two EBM domains ("searching for evidence" and "appraising the evidence") by piloting a questionnaire containing 16 relevant items, and objectively assessed their competence in EBM using an adapted version of the Fresno test, a validated tool. We correlated the matching components between our questionnaire and the Fresno test using Pearson's product-moment correlation.ResultsForty-five out of 72 students in the cohort (62.5%) participated by completing the questionnaire and the adapted Fresno test concurrently. In general, our students perceived themselves as moderately competent in most items of the questionnaire. They rated themselves on average 6.34 out of 10 (63.4%) in "searching" and 44.41 out of 57 (77.9%) in "appraising". They scored on average 26.15 out of 60 (43.6%) in the "searching" domain and 57.02 out of 116 (49.2%) in the "appraising" domain in the Fresno test. The correlations between the students' self-rating and their performance in the Fresno test were poor in both the "searching" domain (r = 0.13, p = 0.4) and the "appraising" domain (r = 0.24, p = 0.1).ConclusionsThis study provides supporting evidence that at the undergraduate level, self-perceived competence in EBM, as measured using our questionnaire, does not correlate well with objectively assessed EBM competence measured using the adapted Fresno test.Study registrationInternational Medical University, Malaysia, research ID: IMU 110/06

An Evidence-based Medicine Curriculum Implemented in Journal Club Improves Resident Performance on the Fresno Test

“…The test was validated in a cohort of family medicine residents and faculty members, and has subsequently been applied either in its original form or a modified form to medical students, physical therapists, occupational therapists, internal medicine residents, psychiatry residents, and pediatric residents (6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18)(19)(20)(21). It has not been independently validated in EM residents, but due to the general nature of the questions, we do not expect that it would perform differently in this cohort.…”

Effect of simulated emergency skills training and assessments on the competence and confidence of medical students

“…[4] Students' inflation of their abilities might be caused by ignorance rather than arrogance; [15] such exaggerated judgements might be the result of an absence of feedback or failure to incorporate feedback into self-perception. [5] Students tended to overestimate their own abilities. High-quality feedback [15] could act as an antidote to such inaccurate self-assessment.…”

“…Several studies have shown that medical students' self-perceived competence correlates poorly with objectively assessed competence. [5] Apart from inadequate self-assessment skills, biased selfevaluation in applied settings can also be ascribed to the overconfidence phenomenon. 'We don't know what we know, but we are confident we do … Not only are we wrong, but we are confident that we are right!'…”

Obstetrics and Gynecology Ultrasound and Evidence-based Medicine in Medical Education

“…Most medical schools include EBM in their curricula but without standardized implementation. 15,16 In a recent comprehensive literature review of EBM training in undergraduate medical education, Maggio et al have found several deficiencies in the current teaching strategies, such as knowledge gap and evaluation changes in practice. 17 The authors also pointed out that all 20 reviewed literature sources lacked robust outcome measures to determine the efficacy of EBM integration in the curricula.…”