Validity, reliability and feasibility of assessment of cilinical reasoning of medical students by observation versus post encounter assessment in a clinical practice setting.

Abstract

Background The assessment of clinical reasoning by medical students in clinical practice is very difficult. Partly this is because the fundamental mechanisms of clinical reasoning are difficult to uncover and when known, hard to observe and interpret. We developed an observation tool to assess the clinical reasoning ability of medical students during clinical practice. The observation tool consists of an 11-item observation rating form. The validity, reliability and feasibility of this tool were verified among medical students during the internal medicine clerkship and compared to a post-encounter rating tool. Results Six raters assessed each the same 15 student patient encounters. The internal consistency (Cronbach’s alfa) for the observation rating tool (ORT) was 0.87 (0.71-0.84) and the 5-item post encounter rating tool (PERT) was 0.81 (0.71-0.87). The intraclass-correlation coefficient for single measurements was poor for both the ORT; 0.32 (p<0.001) as well as the PERT; 0.36 (p<0.001). The G and D-study showed that 6 raters are required to achieve a G-coefficient of > 0.7 for the ORT and 7 raters for the PERT. The largest sources of variance are the interaction between raters and students. There was a correlation between the ORT and PERT of 0.53 (p=0.04) Conclusions The ORT and PERT are both feasible, valid and reliable instruments to assess students’ clinical reasoning skills in clinical practice.