A pilot study to test the effectiveness of an innovative interprofessional education assessment strategy

Abstract

The goals of this quasi-experimental pilot study were to test an assessment tool designed to evaluate students' teamwork skills, and to assess the effectiveness of an interprofessional education (IPE) course. Participants were health professional students (physical therapy, pharmacy, dental and osteopathic medicine), 24 of whom were second-year students who had previously taken part in an IPE course (experimental group), and 22 of whom were third years that had not (control group). Students interacted with a standardized patient and her son during an asynchronous Team Objective Structured Clinical Exam (TOSCE), after which they were scored on their teamwork skills using newly designed teamwork rating scales. Cronbach Alpha calculations suggest that the rating scales are reliable when rater scores are aggregated (0.81). Pearson coefficient calculations determined that teamwork scores of live raters and video raters were significantly correlated (p < 0.0001) suggesting good consistency across these raters, and the experimental group performed significantly better (p = 0.0003) than the control group suggesting that the IPE curriculum is successfully equipping students with teamwork skills. The results of this study contribute to the much needed IPE assessment literature, and suggest that teamwork skills can be taught and effectively assessed using this new rating scale.

When less is more: validating a brief scale to rate interprofessional team competencies

“…Our finding that students who reported prior IPE experience scored higher compared to students reporting none, confirms observations from another study [37] and provides support for the efficacy of IPE for improving team behaviors. We speculate that the abbreviated scale is sensitive to performance differences between groups and can potentially function to evaluate the impact of new IPE curricula.…”

How common are experimental designs in medical education? Findings from a bibliometric analysis of recent dissertations and theses

“…Only 6.12% of these projects involved randomized experiments, a rate that is only slightly higher than a typical dissertation or thesis in the field of education. [4] These results support the notion that research in medical education has far more in common with [5] E D EdD Education Gucev [6] E D EdD Education Miller Juve [7] E D EdD Education Best [8] Q D PhD Engineering Hess [9] Q D DPH Public health McCoy [10] E D EdD Education Buck [11] Q D PhD Psychology Cheung [12] E T MS Medicine Wee [13] E D PhD Medicine Latham [14] E D EdD Education Carney [15] Q T MS Medicine Ford [16] Q D EdD Education Varthis [17] E D PhD Education Hill [18] E T MS Medicine E: Experimental design, Q: Quasi-experimental design, D: Dissertation, T: Thesis, DPH: Doctorate of public health the methodological conventions of education and the social and behavioral sciences than those of the medical and health sciences.…”

Team-Based Decision-Making in an Objective Structured Clinical Examination (OSCE)

“…15 Although no standard assessment context exists for collaborative decision-making, Objective Structured Clinical Examination (OSCE) have been validated as a method to evaluate behavioural competencies, 18 and have been utilized in the pre-licensure setting to evaluate healthcare students' competency in interprofessional collaboration. [19][20][21][22] Emmert & Cai 19 and Zaudke et al 22 utilized team-based OSCE formats wherein students of different health disciplines interacted with a standardized patient asynchronously and synchronously, respectively. Although these studies demonstrate the utility of team-based OSCE in evaluating students' behavioural competency in interprofessional teamwork and communication, use of this OSCE design may be limited by logistical barriers, such as aligning schedules of students and assessors from different healthcare disciplines.…”

“…Domac et al 25 and Bzowyckyj et al 26 describe reflective journaling as a favourable method to assess students' reported experiences related to IPE and intercollaborative practice; however, it is more so a measure of attitude or affective learning, not of behavioural competence. 13,15 OSCE-based evaluations, as utilized to assess interprofessional collaborative practices of students from one 20 or more than one 19,22 healthcare profession, measure behavioural outcomes 13,15 and "transfer of knowledge, skills and/or attitudes." 13 Whether or not affective learning and attitudinal changes translate into long-term changes in associated behaviours has not been well-described in the literature.…”