Evaluation of the team performance observation tool with targeted behavioral markers in simulation-based interprofessional education

Abstract

The primary aim of this study was to decrease the subjectivity of the Team Performance Observation Tool (TPOT) and determine the psychometric properties of this tool when using scenario-specific targeted behavioral markers (TBMs). We used a convenience sample of 47 physical therapy and 25 nursing students at an academic medical center who were organized in interprofessional teams of three to care for a simulated patient. The TPOT demonstrated satisfactory validity and reliability with the use of TBMs. We demonstrated significant correlations between the TPOT overall rating and two scenario-specific outcomes: (1) a negative correlation between the TPOT overall rating and the number of medical errors committed by the 24 teams (r = -0.531, p = 0.008) and (2) a positive correlation between the TPOT overall rating and a time-based functional outcome (r = 0.803, p < 0.001). We demonstrated substantial test-retest reliability (kappa = 0.707, p < 0.001), inter-rater reliability (kappa = 0.730), and good internal consistency reliability (Cronbach's α = 0.921). The results of our study support the use of scenario-specific TBMs to enhance the reliability and validity of the TPOT for use in the evaluation of team-based simulation scenarios. Further evaluation of the TPOT with TBMs from other simulation and training contexts is warranted.

Assessment tools in pre‐licensure interprofessional education: A systematic review, quality appraisal and narrative synthesis

“…Analysis of included articles demonstrated that the nursing profession was the most frequently represented (n = 27), 22‐48 followed by medicine (n = 20), 24,26,27,29,30,32,34‐36,38,40‐44,46‐50 and pharmacy (n = 14) 22,24‐26,28,29,31,41,43,44,50‐53 . A large number of studies were undertaken in the United States of America (n = 24), 22‐25,28‐33,36‐41,45,49‐52,54‐56 with a smaller number from Australia (n = 4), 34,43,57,58 Canada (n = 3) 53,59,60 and the United Kingdom (n = 2) 42,47 (see Appendix ). Acquisition of knowledge was the most commonly assessed outcome (n = 17), 24,26‐29,32,33,35,38,39,43‐45,49,53,55,57 followed by behaviour change (n = 17), 22,25,30‐32,36,37,40,41,44,45,50,55,56,58‐60 benefit to patients (n = 11) 23,34,39,42,46‐48,51,52,54 and acquisition of skills (n = 3) 25,30,31 .…”

“…There was significant heterogeneity in IPE interventions reported in the included studies. The majority of these studies delivered IPE activities in classroom or simulated settings (n = 28) 22,23,25‐33,35‐38,40,41,43‐45,49,50,53,54,56‐59 rather than in clinical/health service settings or community setting (n = 10). IPE interventions varied in their length and episodic nature, where the majority presented one‐off sessions (n = 19), 22,23,25‐29,31,32,35,38,40,41,49,51,53,54,56,57 a short series of activities (n = 8) 36,39,42‐44,46,47,55 and some reported interventions that were of a significant longitudinal nature (months to years) (n = 10) 24,30,33,34,36,38,48,50,52,59 .…”

“…Seventeen studies focused on a change in behaviour following the intervention and were assessed in a range of ways. Twelve tools with validity evidence that measured behaviour are presented in Table 1 22,25,30,31,36,37,40,41,50,56,58,60 and other studies measuring behaviour used a standardised patient checklist (n = 1), 56 a team performance evaluation by faculty (n = 1), 29 a standardised grading rubric based on OSCE evaluation (n = 1) 55 or a 100 mm visual analogue (VAS) clinical decision‐making scale (n = 1) 44…”

Improving Teamwork and Communication through the Use of TeamSTEPPS

“…The learner was able to rate the performance of the nurse leaders on a five-point Likert Scale of one to five, which is very poor to excellent respectively. Chao Zhang and associates demonstrated test-retest reliability of 0.707 [6]. One unintended result of the huddle routine was how serious the leaders took the initiative to optimize the huddle.…”

Operationalization of bandura’s social learning theory to guide interprofessional simulation

“…[4,8,9] Interprofessional practice is a key component for effective communication, improving patient care, increasing understanding between professions, reducing fragmented care, and improving patient safety. [5,[10][11][12][13][14][15] Inclusion of simulation enhanced interprofessional education (IPE) or interprofessional simulation (IPS) can allow health professionals to see the health care team's collaborative impact on patient care. [7,16] In the clinical setting, health care professionals are expected to work in teams, yet, there is limited academic exposure to other allied health students and little preparation is done in traditional classrooms to practice with other allied health students; a gap in curriculum that supports an interprofessional approach exists.…”