Ratings of performance in multisource feedback: comparing performance theories of residents and nurses

Abstract

Background Multisource feedback (MSF) is increasingly being used to assess trainee performance, with different assessor groups fulfilling a crucial role in utility of assessment data. However, in health professions education, research on assessor behaviors in MSF is limited. When assessing trainee performance in work settings, assessors use multidimensional conceptualizations of what constitutes effective performance, also called personal performance theories, to distinguish between various behaviors and sub competencies., This may not only explain assessor variability in Multi Source Feedback, but also result in differing acceptance (and use) of assessment data for developmental purposes. The purpose of this study was to explore performance theories of various assessor groups (residents and nurses) when assessing performance of residents. Methods A constructivist, inductive qualitative research approach and semi-structured interviews following MSF were used to explore performance theories of 14 nurses and 15 residents in the department of internal medicine at Aga Khan University (AKU). Inductive thematic content analysis of interview transcripts was used to identify and compare key dimensions in residents’ and nurses’ performance theories used in evaluation of resident performance. Results Seven major themes, reflecting key dimensions of assessors’ performance theories, emerged from the qualitative data, namely; communication skills, patient care, accessibility, teamwork skills, responsibility, medical knowledge and professional attitude. There were considerable overlaps, but also meaningful differences in the performance theories of residents and the nurses, especially with respect to accessibility, teamwork and medical knowledge. Conclusion Residents’ and nurses’ performance theories for assessing resident performance overlap to some extent, yet also show meaningful differences with respect to the performance dimensions they pay attention to or consider most important. In MSF, different assessor groups may therefore hold different performance theories, depending on their role. Our results further our understanding of assessor source effects in MSF. Implications of our findings are related to implementation of MSF, design of rating scales as well as interpretation and use of MSF data for selection and performance improvement.

A Unique Lens: Understanding What Nurses Are Best Positioned to Assess About Residents

“…While residents are often assessed on their medical knowledge by supervising attendings and peers, both nurse and resident participants felt that nurses should not be asked to assess residents on their medical knowledge or clinical decision-making, which also aligns with prior work. 20 A novel finding in our study was that participants said that nurses might be able to provide feedback on behalf of the patients and families, acting as a proxy for the family experience. Patient feedback adds a new perspective that differs from that of faculty evaluations 21 ; however, patient feedback is the most difficult perspective to capture in multisource feedback.…”

Multi-source feedback following simulated resuscitation scenarios: a qualitative study

“…24 Our finding of different areas of focus for each assessor group is in keeping with this statement and prior research in this field. 25 The feasibility and implementation of MSF into a program of assessment will be challenging and is an active area of research at our institutions. Future work will investigate the use of MSF only for certain key clinical scenarios or partitioning the focus of feedback amongst assessor groups to maximize the value of the feedback and minimize the assessment burden placed on any single assessor group.…”

Bias, Halo Effect and Horn Effect: A Systematic Literature Review

“…Multisource feedback (MSF) is increasingly used to evaluate trainee performance, with different evaluators fulfilling a vital role in the use of evaluation data. (Tariq et al, 2020).…”