The Simulation Team Assessment Tool (STAT): Development, reliability and validation

Reid, Jennifer; Stone, Kimberly; Brown, Julie C.; Caglar, Derya; Kobayashi, Ana; Lewis‐Newby, Mithya; Partridge, Rebecca; Seidel, Kristy; Quan, Linda

doi:10.1016/j.resuscitation.2011.12.012

Cited by 78 publications

(44 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Twelve (16 %) tools measured participants' teamwork skills. Teamwork skills such as leadership, communication and crisis management were assessed through simulation; 41,138,209 however, other tools involved direct observation of skills in actual practice settings, such as medical residents' abilities to lead ward teams 213 and palliative care physicians' communication in team meetings. 195 Behaviors of students, residents/fellows, or practicing physicians were reported outcomes for ten (14 %) tools.…”

Section: Description and Validity Evidence For Teamwork Assessment Toolsmentioning

confidence: 99%

Teamwork Assessment in Internal Medicine: A Systematic Review of Validity Evidence and Outcomes

et al. 2013

View full text Add to dashboard Cite

OBJECTIVE: Valid teamwork assessment is imperative to determine physician competency and optimize patient outcomes. We systematically reviewed published instruments assessing teamwork in undergraduate, graduate, and continuing medical education in general internal medicine and all medical subspecialties. DATA SOURCES: We searched MEDLINE, MEDLINE In-process, CINAHL and PsycINFO from January 1979 through October 2012, references of included articles, and abstracts from four professional meetings. Two content experts were queried for additional studies. STUDY ELIGIBILITY: Included studies described quantitative tools measuring teamwork among medical students, residents, fellows, and practicing physicians on single or multi-professional (interprofessional) teams. STUDY APPRAISAL AND SYNTHESIS METHODS:Instrument validity and study quality were extracted using established frameworks with existing validity evidence. Two authors independently abstracted 30 % of articles and agreement was calculated. RESULTS: Of 12,922 citations, 178 articles describing 73 unique teamwork assessment tools met inclusion criteria. Interrater agreement was intraclass correlation coefficient 0.73 (95 % CI 0.63-0.81). Studies involved practicing physicians (142, 80 %), residents/fellows (70, 39 %), and medical students (11, 6 %). The majority (152, 85 %) assessed interprofessional teams. Studies were conducted in inpatient (77, 43 %), outpatient (42, 24 %), simulation (37, 21 %), and classroom (13, 7 %) settings. Validity evidence for the 73 tools included content (54, 74 %), internal structure (51, 70 %), relationships to other variables (25, 34 %), and response process (12, 16 %). Attitudes and opinions were the most frequently assessed outcomes. Relationships between teamwork scores and patient outcomes were directly examined for 13 (18 %) of tools. Scores from the Safety Attitudes Questionnaire and Team Climate Inventory have substantial validity evidence and have been associated with improved patient outcomes. LIMITATIONS: Review is limited to quantitative assessments of teamwork in internal medicine. CONCLUSIONS:There is strong validity evidence for several published tools assessing teamwork in internal medicine. However, few teamwork assessments have been directly linked to patient outcomes. 3,4 Recognizing this, the Institute of Medicine, the Joint Commission, the Agency for Healthcare Research and Quality (AHRQ), and others have made teamwork a top priority in their recommendations for improving healthcare. [5][6][7][8][9] Teamwork is also prominently positioned within the American Board of Internal Medicine (ABIM) requirements for maintenance of certification for internists, 10 as well as the Accreditation Council for Graduate Medical Education's core competencies, 11 milestones, 12 and medical student competencies. 13 As such, every physician at the undergraduate, graduate, and continuing professional level must demonstrate competency in teamwork.While there is broad agreement on the imperative to improve teamwork, there is littl...

show abstract

Section: Description and Validity Evidence For Teamwork Assessment Toolsmentioning

confidence: 99%

Teamwork Assessment in Internal Medicine: A Systematic Review of Validity Evidence and Outcomes

et al. 2013

View full text Add to dashboard Cite

show abstract

“…The sources of funding were as follows: five federal (two NIH, one Canada, and one Taiwan), five universitysponsored, two industry-supported, and two organizationally funded. Research methodology included 17 surveys (36%), 17,19,21,25,30,31,33,37,40,42,46,[50][51][52]54,60,61 15 (31%) observational analyses, [14][15][16]22,23,27,32,33,38,44,47,49,56,57,59 and three (6%) qualitative methodology studies. 19,28,45 There were only 12 (25%) with an experimental or quasi-experimental study design, 20,24,26,35,36,39,41,43,48,53,55,57 with five of the highlighted articles using this rigorous design.…”

Section: Trends In Medical Education Research In 2012mentioning

confidence: 99%

“…Three (6%) appeared in medical education journals, and nine (19%) were published in journals ranging from surgery 33,42,44,54 to psychiatry. 31,52 Others included palliative medicine, 26 critical care, 56 and simulation. 38 Most articles (81%) had authors with an EM affiliation.…”

Section: Trends In Medical Education Research In 2012mentioning

confidence: 99%

“…33,41,43 Seven of these focused on learner satisfaction or self-assessment regarding the simulation experience, while nine were outcomes based. 22,32,33,38,39,41,43,56,57 Five articles studied ultrasound in EM, 21,47,49,59,61 and two employed visual technology. 55,58 A positive trend this year was the emergence of more competency-based studies, which reflect a higher order of research outcomes in medical education, based on the Kirkpatrick model.…”

Section: Trends In Medical Education Research In 2012mentioning

confidence: 99%

“…18,33,41,45,46 Pediatric topics accounted for nine (19%) of the articles reviewed. [20][21][22]24,40,41,55,56,58 The two that are in the highlighted section used technology as the basis of their studies, and both received funding. 41,55 Each of the articles with a pediatric focus had the primary or supporting author with specialized pediatric training.…”

Section: Trends In Medical Education Research In 2012mentioning

confidence: 99%

See 2 more Smart Citations

Critical Appraisal of Emergency Medicine Education Research: The Best Publications of 2012

Lin

Fisher

Coates

et al. 2014

Academic Emergency Medicine

View full text Add to dashboard Cite

Objectives: The objective was to critically appraise and highlight medical education research published in 2012 that was methodologically superior and whose outcomes were pertinent to teaching and education in emergency medicine (EM).Methods: A search of the English language literature in 2012 querying Education Resources Information Center (ERIC), PsychInfo, PubMed, and Scopus identified EM studies using hypothesis-testing or observational investigations of educational interventions. Two reviewers independently screened all of the publications and removed articles using established exclusion criteria. This year, publications limited to a single-site survey design that measured satisfaction or self-assessment on unvalidated instruments were not formally reviewed. Six reviewers then independently ranked all remaining publications using one of two scoring systems depending on whether the study methodology was primarily qualitative or quantitative. Each scoring system had nine criteria, including four related to methodology, that were chosen a priori, to standardize evaluation by reviewers. The quantitative study scoring system was used previously to appraise medical education published annually in 2008 through 2011, while a separate, new qualitative study scoring system was derived and implemented consisting of parallel metrics.Results: Forty-eight medical education research papers met the a priori criteria for inclusion, and 33 (30 quantitative and three qualitative studies) were reviewed. Seven quantitative and two qualitative studies met the criteria for inclusion as exemplary and are summarized in this article. In this fifth installment of the annual critical appraisal series, the same six reviewers used previously published criteria to critically analyze and rank the EM education research from 2012. The focus of this article is to review and highlight the methodologically superior studies that are pertinent to teaching and education in EM. Trends in EM education research over the past 5 years are summarized. We hope that this paper will serve as a valuable resource for EM educators and researchers invested in the scholarship of teaching. 2 METHODS Article IdentificationA medical librarian (LM) performed the literature search in the medical and social sciences literature domains and supplied medical subject heading (MeSH) and keyword terms. MEDLINE was searched through PubMed using a Boolean search strategy that incorporated the following MeSH terms: emergency medicine and medical education, medical student, internship and residency, teaching rounds, undergraduate medical education, graduate medical education, and continuing medical education. Keyword variants for the MeSH terms were included in the search for comprehensiveness. Boolean searches of other databases, including Scopus ("medical education" and emergency), Education Resources Information Center (ERIC; emergency medicine), and PsychInfo (emergency medicine and education) were performed using keyword searching and where possible using the da...

show abstract

Effectiveness of a Novel Tablet Application in Reducing Guideline Deviations During Pediatric Cardiac Arrest

Corazza,

Arpone,

Tardini

et al. 2023

JAMA Netw Open

View full text Add to dashboard Cite

ImportanceDeviations from international resuscitation guidelines during the management of pediatric cardiac arrest are frequent and affect clinical outcomes. An interactive tablet application (app), PediAppRREST, was developed to reduce guideline deviations during pediatric cardiac arrest.ObjectiveTo assess the effectiveness of PediAppRREST in improving the management of simulated in-hospital pediatric cardiac arrest.Design, Setting, and ParticipantsThis multicenter 3-group simulation-based randomized clinical trial was conducted from September 2020 to December 2021 at 4 Italian university hospitals (Padua, Florence, Rome, Novara). Participants included residents in pediatrics, emergency medicine, and anesthesiology. Analyses were conducted as intention-to-treat. Data were analyzed from January to June 2022.InterventionsTeams were randomized to 1 of 3 study groups: an intervention group that used the PediAppRREST app; a control group that used a paper-based cognitive aid, the Pediatric Advanced Life Support (PALS) pocket card; and a control group that used no cognitive aids. All the teams managed the same standardized simulated scenario of nonshockable pediatric cardiac arrest.Main Outcomes and MeasuresThe primary outcome was the number of deviations from guidelines, measured by a 15-item checklist based on guideline recommendations. The main secondary outcomes were quality of chest compressions, team clinical performance (measured by the Clinical Performance Tool), and perceived team leader’s workload. Study outcomes were assessed via video reviews of the scenarios.ResultsOverall 100 teams of 300 participants (mean [SD] age, 29.0 [2.2] years; 195 [65%] female) were analyzed by intention-to-treat, including 32 teams randomized to the PediAppRREST group, 35 teams randomized to the PALS control group, and 33 teams randomized to the null control group. Participant characteristics (210 pediatric residents [70%]; 48 anesthesiology residents [16%]; 42 emergency medicine residents [14%]) were not statistically different among the study groups. The number of deviations from guidelines was significantly lower in the PediAppRREST group than in the control groups (mean difference vs PALS control, −3.0; 95% CI, −4.0 to −1.9; P &lt; .001; mean difference vs null control, −2.6; 95% CI, −3.6 to −1.5; P &lt; .001). Clinical Performance Tool scores were significantly higher in the PediAppRREST group than control groups (mean difference vs PALS control, 1.4; 95% CI, 0.4 to 2.3; P = .002; mean difference vs null control, 1.1; 95% CI, 0.2 to 2.1; P = .01). The other secondary outcomes did not significantly differ among the study groups.Conclusions and RelevanceIn this randomized clinical trial, the use of the PediAppRREST app resulted in fewer deviations from guidelines and a better team clinical performance during the management of pediatric cardiac arrest.Trial RegistrationClinicalTrials.gov Identifier: NCT04619498

show abstract

The Simulation Team Assessment Tool (STAT): Development, reliability and validation

Cited by 78 publications

References 30 publications

Teamwork Assessment in Internal Medicine: A Systematic Review of Validity Evidence and Outcomes

Teamwork Assessment in Internal Medicine: A Systematic Review of Validity Evidence and Outcomes

Critical Appraisal of Emergency Medicine Education Research: The Best Publications of 2012

Effectiveness of a Novel Tablet Application in Reducing Guideline Deviations During Pediatric Cardiac Arrest

Contact Info

Product

Resources

About