How biased are you? The effect of prior performance information on attending physician ratings and implications for learner handover

Shaw, Tammy; Wood, Timothy; Touchie, Claire; Pugh, Debra; Humphrey‐Murto, Susan

doi:10.1007/s10459-020-09979-6

Cited by 22 publications

(22 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While understanding supervisor perspectives is critical to successful implementation, discussions regarding learner handover would benefit from robust evidence regarding the observed benefits and harms of learner handover as well as factors influencing their likelihood and extent. The majority of available experimental studies can be found within the educational, social, and organizational psychology literature [5,[35][36][37][38], with a small, but growing, lit-erature in health professions education with mixed results [17,39]. Further studies are needed to understand the potential positive and negative impacts of learner handover in medical education settings.…”

Section: Future Directionsmentioning

confidence: 99%

Learner handover: Perspectives and recommendations from the front-line

Gumuchian¹,

Pal²,

Young³

et al. 2020

Perspect Med Educ

View full text Add to dashboard Cite

Introduction Current medical education models increasingly rely on longitudinal assessments to document learner progress over time. This longitudinal focus has re-kindled discussion regarding learner handover—where assessments are shared across supervisors, rotations, and educational phases, to support learner growth and ease transitions. The authors explored the opinions of, experiences with, and recommendations for successful implementation of learner handover among clinical supervisors. Methods Clinical supervisors from five postgraduate medical education programs at one institution completed an online questionnaire exploring their views regarding learner handover, specifically: potential benefits, risks, and suggestions for implementation. Survey items included open-ended and numerical responses. The authors used an inductive content analysis approach to analyze the open-ended questionnaire responses, and descriptive and correlational analyses for numerical data. Results Seventy-two participants completed the questionnaire. Their perspectives varied widely. Suggested benefits of learner handover included tailored learning, improved assessments, and enhanced patient safety. The main reported risk was the potential for learner handover to bias supervisors’ perceptions of learners, thereby affecting the validity of future assessments and influencing the learner’s educational opportunities and well-being. Participants’ suggestions for implementation focused on who should be involved, when and for whom it should occur, and the content that should be shared. Discussion The diverse opinions of, and recommendations for, learner handover highlight the necessity for handover to maximize learning potential while minimizing potential harms. Supervisors’ suggestions for handover implementation reveal tensions between assessment-of and for-learning.

show abstract

Section: Future Directionsmentioning

confidence: 99%

Learner handover: Perspectives and recommendations from the front-line

Gumuchian¹,

Pal²,

Young³

et al. 2020

Perspect Med Educ

View full text Add to dashboard Cite

show abstract

“…It may be for these reasons that faculty were more likely to report being "uncertain" when rating students they were unfamiliar with. However, awareness of poor past performance has also been shown to negatively in uence current assessments (47), which may explain the differences in entrustment assessments seen in this study (Table 1). Relatedly, "affective response" was typically disappointment that a student did not meet expectations.…”

Section: Discussionmentioning

confidence: 74%

Pre-Clerkship EPA Assessments: A Thematic Analysis of Rater Cognition

Meyer

Harvey

Durning

et al. 2021

Preprint

View full text Add to dashboard Cite

Background. Entrustable Professional Activities (EPAs) assessments measure learners’ competence with an entrustment or supervisory scale. Designed for workplace-based assessment EPA assessments have also been proposed for undergraduate medical education (UME), where assessments frequently occur outside the workplace and may be less intuitive, raising validity concerns. This study explored how assessors make entrustment determinations in UME, to include the impact of longitudinal student-assessor relationships.Methods. A qualitative approach using think-alouds was employed. Assessors assessed two students (familiar and unfamiliar) completing a history and physical exam using a supervisory scale and then thought-aloud after each assessment. We conducted a thematic analysis of assessors’ response processes and compared them based on their familiarity with a student.Results. Four themes and fifteen subthemes were identified. The most prevalent theme related to “student performance.” The other three themes included “frame of reference,” “assessor uncertainty,” and “the patient.” “Previous student performance” and “affective reactions” were subthemes more likely to inform scoring when faculty were familiar with a student, while unfamiliar faculty were more likely to reference “self” and “lack confidence in their ability to assess.”Conclusions. Student performance appears to be assessors’ main consideration for all students, providing some validity evidence for the response process in EPA assessments. Several problematic themes could be addressed with faculty development while others appear to be inherent to entrustment and may be more challenging to mitigate. Differences based on assessor familiarity with student merits further research on how trust develops over time.

show abstract

“…This shows that the inherent effects of ILH cannot simply be attributed to assimilation bias or hive thinking 3,9,11,13 . There are many facets to how information is transferred and accepted by individuals and therefore require deeper understanding of the environment the handover practices take place.…”

Section: Personal Biasmentioning

confidence: 89%

“…The ability of LH to bias future teachers has shown up in multiple studies 9,12,13 . However, the exact nature of such a bias seems to be disputed as different studies show different effects.…”

Section: Introductionmentioning

confidence: 99%

Effects of Informal Learner Handover in Clinical Dental Education

Shahid

Khan

Arooj

2022

Preprint

View full text Add to dashboard Cite

Purpose:Learner Handover (LH) protocols to enable guided sharing of student performance amongst faculty can be especially useful in the transition from lab-to-clinic work in undergraduate dental training due to the build-up of skills required and a small, more manageable sized student body. However, there is an ongoing debate as to its benefits, disadvantages, and outright usefulness. Moreover, the extent of existing ‘Informal Learner Handover’ (ILH) in the form of faculty discussions is not actively researched. Exploring the nature of ILH may provide additional context to stakeholders. Furthermore, it can also provide insight into the nature of Learner Handover bias so that they can be minimized when designing a Formal Learner Handover (FLH) Protocol.Methods:A grounded theory study design was selected to review and identify relevant patterns in a series of semi-structured interviews and focus group discussions from January to March 2022. Voluntary participation of 16 active clinical dental faculty of a variety of designations and seniority were included in the study. No opinion was discarded.Results:Four themes were identified after the thematic analysis, with twelve subthemes. These were: 1) effect on Behavior (Direct Interaction, Disability Aid, and Mindset), 2) effect on Student Expectation (Knowledge base, Pre-formed Opinions, and Student Competence), 3) effect on teaching approach (Refresher, Progress of clinical rotations, Student capability management, and modifications), and 4) effect on feedback practices (Group problems identified, and communication).Conclusions:ILH has a minor impact on faculty interactions with their students in clinical dental training. Other factors that contribute to the student’s ‘academic reputation’ provide more of an impact in faculty perceptions and interactions with their students.

show abstract

How biased are you? The effect of prior performance information on attending physician ratings and implications for learner handover

Cited by 22 publications

References 42 publications

Learner handover: Perspectives and recommendations from the front-line

Learner handover: Perspectives and recommendations from the front-line

Pre-Clerkship EPA Assessments: A Thematic Analysis of Rater Cognition

Effects of Informal Learner Handover in Clinical Dental Education

Contact Info

Product

Resources

About