BackgroundCurrent scales for interprofessional team performance do not provide adequate behavioral anchors for performance evaluation. The Team Observed Structured Clinical Encounter (TOSCE) provides an opportunity to adapt and develop an existing scale for this purpose. We aimed to test the feasibility of using a retooled scale to rate performance in a standardized patient encounter and to assess faculty ability to accurately rate both individual students and teams.MethodsThe 9-point McMaster-Ottawa Scale developed for a TOSCE was converted to a 3-point scale with behavioral anchors. Students from four professions were trained a priori to perform in teams of four at three different levels as individuals and teams. Blinded faculty raters were trained to use the scale to evaluate individual and team performances. G-theory was used to analyze ability of faculty to accurately rate individual students and teams using the retooled scale.ResultsSixteen faculty, in groups of four, rated four student teams, each participating in the same TOSCE station. Faculty expressed comfort rating up to four students in a team within a 35-min timeframe. Accuracy of faculty raters varied (38–81% individuals, 50–100% teams), with errors in the direction of over-rating individual, but not team performance. There was no consistent pattern of error for raters.ConclusionThe TOSCE can be administered as an evaluation method for interprofessional teams. However, faculty demonstrate a ‘leniency error’ in rating students, even with prior training using behavioral anchors. To improve consistency, we recommend two trained faculty raters per station.
Background: There is a need for validated and easy-to-apply behavior-based tools for assessing interprofessional team competencies in clinical settings. The seven-item observer-based Modified McMaster-Ottawa scale was developed for the Team Objective Structured Clinical Encounter (TOSCE) to assess individual and team performance in interprofessional patient encounters.Objective: We aimed to improve scale usability for clinical settings by reducing item numbers while maintaining generalizability; and to explore the minimum number of observed cases required to achieve modest generalizability for giving feedback.Design: We administered a two-station TOSCE in April 2016 to 63 students split into 16 newly-formed teams, each consisting of four professions. The stations were of similar difficulty. We trained sixteen faculty to rate two teams each. We examined individual and team performance scores using generalizability (G) theory and principal component analysis (PCA).Results: The seven-item scale shows modest generalizability (.75) with individual scores. PCA revealed multicollinearity and singularity among scale items and we identified three potential items for removal. Reducing items for individual scores from seven to four (measuring Collaboration, Roles, Patient/Family-centeredness, and Conflict Management) changed scale generalizability from .75 to .73. Performance assessment with two cases is associated with reasonable generalizability (.73). Students in newly-formed interprofessional teams show a learning curve after one patient encounter. Team scores from a two-station TOSCE demonstrate low generalizability whether the scale consisted of four (.53) or seven items (.55).Conclusion: The four-item Modified McMaster-Ottawa scale for assessing individual performance in interprofessional teams retains the generalizability and validity of the seven-item scale. Observation of students in teams interacting with two different patients provides reasonably reliable ratings for giving feedback. The four-item scale has potential for assessing individual student skills and the impact of IPE curricula in clinical practice settings.Abbreviations: IPE: Interprofessional education; SP: Standardized patient; TOSCE: Team objective structured clinical encounter
PA students participating in a medical Spanish curriculum and SPs show good correlation with an expert faculty rater in assessing Spanish proficiency during an OSCE. Standardized patients demonstrate scoring leniency. The ILR has potential for tracking aggregate student progress and curriculum effectiveness. With training, student self-rating could be used for interval assessment of medical Spanish communication.
To examine concordance between in-room and video faculty ratings of interprofessional behaviors in a standardized team objective structured clinical encounter (TOSCE). In-room and video-rated student performance scores in an interprofessional 2-station TOSCE were compared using a validated 3-point scale assessing six team competencies. Scores for each student were derived from two in-room faculty members and one faculty member who viewed video recordings of the same team encounter from equivalent visual vantage points. All faculty members received the same rigorous rater training. Paired sample t-tests were used to compare individual student scores. McNemar's test was used to compare student pass/fail rates to determine the impact of rating modality on performance scores. In-room and video student scores were captured for 12 novice teams (47 students) with each team consisting of students from four professions (medicine, pharmacy, physician assistant, nursing). Video ratings were consistently lower for all competencies and significantly lower for competencies of roles and responsibilities, and conflict management. Using a criterion of an average score of 2 out of 3 for at least one station for passing, 56% of students passed when rated in-room compared with 20% when rated by video. In-room and video ratings are not equal. Educators should consider scoring discrepancies based on modality when assessing team behaviors.
Purpose This study describes and examines the short- and longer-term impact of a required longitudinal medical Spanish curriculum on physician assistant student preparedness and ability to communicate with patients in Spanish during clinical rotations. Methods Fifty-eight preclinical students participated in an 80-hour curriculum delivered weekly over 3 semesters. Teaching followed a framework of second-language acquisition and included structured grammar and medical vocabulary practice with didactic, interactive, and group assignments. Vocabulary and grammar were assessed with quizzes. Oral proficiency was assessed by faculty with Spanish Objective Structured Clinical Examination (OSCE) stations at midpoint and end using the Interagency Language Roundtable (ILR), a 6-level scale (immediate outcome). Students self-rated proficiency and confidence and evaluated curriculum effectiveness for preparing them to care for Spanish-speaking patients (longer-term outcomes). Results All students passed the written and oral quizzes. Faculty-scored ILR verbal proficiency at the OSCEs increased by a mean level of 0.5 over 6 months. Student self-assessed proficiency improved on average by one level from baseline to 24 months later. Students rated highly curriculum effectiveness, preparedness to communicate in Spanish during clinical rotations, ability to judge when an interpreter was needed, and the importance of medical Spanish to future practice. Conclusions A required integrated longitudinal medical Spanish curriculum was well received. Physician assistant students demonstrated short-term interval progression in Spanish proficiency, with improvements in both faculty and self-rating scores, and readiness to apply the skill to practice. They valued active learning associated with repeated practice with feedback, role playing, and interval assessments throughout the curriculum.
No abstract
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.