Additional tools to boost internal medicine residents’ evaluations

Abdel-Aziz, Yousef; Barnett, William; Altorok, Nezam; Assaly, Ragheb

doi:10.5116/ijme.5c6c.358f

Cited by 1 publication

(1 citation statement)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As noted by Abdel-Aziz et al, 21 resident assessment is extremely complex and should incorporate many different indices in addition to faculty evaluations. However, each information source must be as valid as possible to ensure that resident experiences are tailored appropriately to optimize their skill set development.…”

Section: Discussionmentioning

confidence: 99%

Is It All About the Form? Norm- vs Criterion-Referenced Ratings and Faculty Inter-Rater Reliability

Scielzo

AbdelFattah

Ryder

2023

TOJ

View full text Add to dashboard Cite

Background: Little research to date has examined the quality of data obtained from resident performance evaluations. This study sought to address this need and compared inter-rater reliability obtained from norm-referenced and criterion-referenced evaluation scaling approaches for faculty completing resident performance evaluations. Methods: Resident performance evaluation data were examined from 2 institutions (3 programs, 2 internal medicine and 1 surgery; 426 residents in total), with 4 evaluation forms: 2 criterion-referenced (1 with an additional norm-referenced item) and 2 norm-referenced. Faculty inter-rater reliability was calculated with intraclass correlation coefficients (ICCs) (1,10) for each competency area within the form. ICCs were transformed to z -scores, and 95% CIs were computed. Reliabilities for each evaluation form and competency, averages within competency, and averages within scaling type were examined. Results: Inter-rater reliability averages were higher for all competencies that used criterion-referenced scaling relative to those that used norm-referenced scaling. Aggregate scores of all independent categories (competencies and the items assessing overall competence) for criterion-referenced scaling demonstrated higher reliability ( z =1.37, CI 1.26-1.48) than norm-referenced scaling ( z =0.88, CI 0.77-0.99). Moreover, examination of the distributions of composite scores (average of all competencies and raters for each individual being rated) suggested that the criterion-referenced evaluations better represented the performance continuum. Conclusion: Criterion-referenced evaluation approaches appear to provide superior inter-rater reliability relative to norm-referenced evaluation scaling approaches. Although more research is needed to identify resident evaluation best practices, using criterion-referenced scaling may provide more valid data than norm-referenced scaling.

show abstract

Section: Discussionmentioning

confidence: 99%