Development of a Simulation-Based Interprofessional Teamwork Assessment Tool

Abstract

Background The Accreditation Council for Graduate Medical Education (ACGME) Milestone projects required each specialty to identify essential skills and develop means of assessment with supporting validity evidence for trainees. Several specialties rate trainees on a milestone subcompetency related to working in interprofessional teams. A tool to assess trainee competence in any role on an interprofessional team in a variety of scenarios would be valuable and suitable for simulation-based assessment.Objective We developed a tool for simulation settings that assesses interprofessional teamwork in trainees.Methods In 2015, existing tools that assess teamwork or interprofessionalism using direct observation were systematically reviewed for appropriateness, generalizability, adaptability, ease of use, and resources required. Items from these tools were included in a Delphi method with multidisciplinary pediatrics experts using an iterative process from June 2016 to January 2017 to develop an assessment tool.Results Thirty-one unique tools were identified. A 2-stage review narrowed this list to 5 tools, and 81 items were extracted. Twenty-two pediatrics experts participated in 4 rounds of Delphi surveys, with response rates ranging from 82% to 100%. Sixteen items reached consensus for inclusion in the final tool. A global 4-point rating scale from novice to proficient was developed.Conclusions A novel tool to assess interprofessional teamwork for individual trainees in a simulated setting was developed using a systematic review and Delphi methodology. This is the first step to establish the validity evidence necessary to use this tool for competency-based assessment. MethodsOur team included simulation experts from pediatric emergency medicine, pediatric hospital medicine, and graduate medical education from 7 institutions across

Application of a Simulation-Based Interprofessional Teamwork Assessment Tool (SITAT) to Individual Student Performance in a Team-Based Simulation

“…Historically, groups of students have been assessed as a whole team. 8 We postulate that if there are one or more very high performers among the group, the team competency is likely a reflection of those individuals rather than the whole team. This may potentially result in a missed opportunity to evaluate the more reserved or less confident learners because of limited data on their individual contributions, although no studies to date have determined if this is accurate.…”

“…In 2019, Bismilla et al 8 developed a tool for the measurement of individual competence on an interprofessional team, in any role during simulation-based activities. The tool was created based on the results of a 2016 survey of both simulation experts and pediatric program director recognizing a need for competencybased assessment instruments to be utilized in different settings, specifically simulation.…”

“…; and (4) did it meet AGME criteria of ease of use. 8 Upon initial review, the qualified tools were them moved to Phase 2, where they were rated based on their difficulty of adaption and scored on a scale from 1, no adaptation to use to 7, extensive adaptation necessary. All tools with an adaptability rating greater than 3 were retained.…”

“…7 The limited number of instruments that allow external raters to provide individual student assessment have numerous items to complete, making it difficult to deploy in a timed simulation and lacking high reliability. 7 Bismilla et al 8 used Delphi methodology to develop and achieve expert consensus (content validity) on a 16-item instrument, the Simulation-Based Interprofessional Teamwork Assessment Tool (SITAT), for assessing an individual on an interprofessional team. This tool aims to fill gaps in simulationbased education by providing individualized team member assessment though a manageable number of items to be completed by an observer.…”

A Decade Later—Progress and Next Steps for Pediatric Simulation Research

“…This led to the development and dissemination of simulation-based assessment tools in multiple competencies identified as difficult to assess using traditional methods. [4][5][6][7] The consensus group recognized that simulation could be used to teach a wide array of learning objectives beyond crisis management. A review of the Association of American Medical College MedEdPortal website's published educational files with the search term "pediatric simulation" revealed 280 total cases, 257 of which were published in the last decade.…”