Difficulties in clinical skills evaluation

Abstract

A case-specific method of evaluating clinical skills is presented. The instrument is described together with its evaluation based on results from ninety-two medical students. The method involves direct observation, by pairs of teachers, of student encounters with simulated patients. The purpose of this paper is to explore the methodology of assessing clinical competence. Three aspects of this are addressed: reliability between teachers acting as observers, and the relationship of both observations of process measures and self-assessments with the students' understanding of the patient. Three aspects of clinical skills were evaluated; interviewing, problem orientation and physical examination. The results showed the observers to be highly reliable in observing physical examination but to have a wide range of reliability across cases in their observations of the other aspects of clinical skill. There was no correlation between both teachers' observations and the students' self-assessments with the students' understanding of the patient. This is a worrying finding. The implications of this study are that evaluation of the process of clinical skills is difficult methodologically and, added to which, it is not clear what relevance these observations have to clinical competence. Rather than effort being devoted to refining instruments to measure the process of clinical skills it would seem preferable to devote energy to assessing students' abilities in diagnosis and management, at least until we understand what observations of clinical skills are measuring.

SPECIAL ARTICLE: Cognitive, Social and Environmental Sources of Bias in Clinical Performance Ratings

“…Research on clinical problem solving, 8 clinical performance assessment (primarily using standardized patient technology), [9][10][11][12] and clinical practice has shown consistently that clinical performance is case specific. Practitioners who excel at diagnosing and managing one clinical problem are not necessarily skillful at diagnosing even closely related clinical situations.…”

A factor analytic study of clinical competence in undergraduate psychiatry

“…Therefore there is little wonder that it is a difficult phenomenon to examine precisely (Newble 1978). However, a reasonable aim is to define the component skills which make up clinical competence so that they can be taught and examined (Harper et al 1983).…”

Curriculum evaluation: status and options

“…However, a major difficulty of the literature is that it tends to be so broad as to be of limited utility in attempting to bring about desired improvement. A large number of studies have been on such topics as clinical and didactic teaching and evaluation, whereas little attention has been given to exploring teachers’perceptions of the curriculum and finding out their perceptions of what has worked well and facilitated the effectiveness of the curriculum (interested readers are referred to Berner et al , 1974; Rippey, 1975; Fox, 1983; Veiga et al , 1983; Harper et al , 1983). The investigators of the present study, therefore, decided to approach teaching staff in an endeavour to obtain their perceptions of the curriculum and their opinions towards selected aspects of medical education.…”