A modified electronic key feature examination for undergraduate medical students: validation threats and opportunities

Abstract

The purpose of our study was the development and validation of a modified electronic key feature exam of clinical decision-making skills for undergraduate medical students. Therefore, the reliability of the test (15 items), the item difficulty level, the item-total correlations and correlations to other measures of knowledge (40 item MC-test and 580 items of German MC-National Licensing Exam, Part II) were calculated. Based on the guidelines provided by the Medical Council of Canada, a modified electronic key feature exam for internal medicine consisting of 15 key features (KFs) was developed for fifth year German medical students. Long menu (LM) and short menu (SM) question formats were used. Acceptance was assessed through a questionnaire. Thirty-seven students from four medical schools voluntarily participated in the study. The reliability of the key feature exam was 0.65 (Cronbach's alpha). The items' difficulty level scores were between 0.3 and 0.8 and the item-total correlations between 0.0 and 0.4. Correlations between the results of the KF exam and the other measures of knowledge were intermediate (r between 0.44 and 0.47) as well as the learners' level of acceptance. The modified electronic KF examination is a feasible and reliable evaluation tool that may be implemented for the assessment of clinical undergraduate training.

Effects of a supplementary final year curriculum on students’ clinical reasoning skills as assessed by key-feature examination

“…Key-feature examinations assess competencies at the 'knows-how' level according to Miller's learning pyramid (Miller 1990). They have proved to be a reliable and valid approach in assessing clinical reasoning skills Fischer et al 2005) and to represent a feasible tool in evaluating internal medicine clerkships (Hatala & Norman 2002). In this study, we were able to achieve reliabilities of 0.71 and 0.75 (Cronbach's alpha).…”

“…The Swiss one-dimensional blueprint for internal medicine was used (Bloch & Burgi 2002) to weight sub-domains in internal medicine in order to ensure equivalent learning objectives for both experimental and comparison groups. The main body of the key-feature examination used in this study was derived and adapted from Fischer et al (2005), whose key-feature items have already proven to be a reliable assessment tool in undergraduate internal medicine education. The development and construction of further key-feature problems was based on the steps recommended by .…”

Assessment Score Comparison of Knowledge and Clinical Reasoning of Medical Student Educated by Bedside or Conference Room Teaching in Internal Medicine Ward

“…Fischer et al believes that the KF Questions is a reliable method for clinical competency evaluation. 10 The KF exam can be designed with varying numbers of questions and is able to distinguish between the experienced and the beginner student. 18 The KF exam is generally seen as an appropriate method of clinical reasoning evaluation and able to predict clinical practice authority in the future, although it is not possible to evaluate all clinical aspects by this method.…”

“…Multiple Choice Questions (MCQ), Key Feature (KF) clinical reasoning and Objective Structured Clinical Examination (OSCE) are main methods for assessing medical students in most academic arenas. 9,10 The impact of educational methods on the outcome of educational assessments has been less studied in educational institutions. A study was conducted to compare exam scores of students after training with one of 2 methods: structured bedside teaching or traditional bedside teaching indicated that the results are not different.…”

Virtual Patients as a Practical Realisation of the E-learning Idea in Medicine

“…Two corollaries were added by Page and Bordage: It focuses on a step in which examinees are most likely to make errors in the resolution of the problem, and it is a difficult aspect in of the identification and management of the problem in practice . A study by Fischer et al introduced a 15-item key feature exam for assessing clinical decision-making skills in undergraduate students (Fischer et al, 2005b)…”