BackgroundIncreased attention on collaboration and teamwork competency development in medical education has raised the need for valid and reliable approaches to the assessment of collaboration competencies in post-graduate medical education. The purpose of this study was to evaluate the reliability of a modified Interprofessional Collaborator Assessment Rubric (ICAR) in a multi-source feedback (MSF) process for assessing post-graduate medical residents’ collaborator competencies.MethodsPost-graduate medical residents (n = 16) received ICAR assessments from three different rater groups (physicians, nurses and allied health professionals) over a four-week rotation. Internal consistency, inter-rater reliability, inter-group differences and relationship between rater characteristics and ICAR scores were analyzed using Cronbach’s alpha, one-way and two-way repeated measures ANOVA, and logistic regression.ResultsMissing data decreased from 13.1% using daily assessments to 8.8% utilizing an MSF process, p = .032. High internal consistency measures were demonstrated for overall ICAR scores (α = .981) and individual assessment domains within the ICAR (α = .881 to .963). There were no significant differences between scores of physician, nurse, and allied health raters on collaborator competencies (F2,5 = 1.225, p = .297, η2 = .016). Rater gender was the only significant factor influencing scores with female raters scoring residents significantly lower than male raters (6.12 v. 6.82; F1,5 = 7.184, p = .008, η 2 = .045).ConclusionThe study findings suggest that the use of the modified ICAR in a MSF assessment process could be a feasible and reliable assessment approach to providing formative feedback to post-graduate medical residents on collaborator competencies.
The neonatal resuscitation skills of 30 third-year medical students were assessed in real time by a face-to-face examiner in the same room as the student, and by a remote examiner located in a separate room using the ANAKIN system. The ANAKIN system combines an instrumented manikin simulator, computer-based assessment and high-bandwidth videoconferencing. The students were assessed while performing a neonatal resuscitation megacode using the ANAKIN system. Students were satisfied with the ANAKIN system as an assessment system and were not intimidated by its use. However, the correlation between the face-to-face and remote examiner's mean total performance assessment scores was 0.27, which was not significant (P=0.14). The results indicated variation between the examiners' performance scores in a number of key technical skill areas. The findings from this study have implications for the use of technology-mediated systems in assessing resuscitation skills. Examiner orientation is critical for individuals using such systems. These persons must be comfortable and confident in using the technology. Interface and design features of the system need to be carefully scrutinized and tested.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.