Evaluation of a Brief Team Training Intervention in Surgery: A Mixed‐Methods Study

Abstract

The aim of this study was to evaluate a brief team training program in relation to teams' observed nontechnical skills (NTSs) in surgery, teams' perceptions of safety culture, and the training implementation. We used mixed methods to analyze structured observations of 179 surgeries, semistructured interviews with surgical team members from four selected surgical specialties, and a survey. There were significant (P < .001) improvements in surgical teams' observed NTSs and in the use of the World Health Organization's Surgical Safety Checklist after participation in the training program. Nonsignificant results included increased perceived safety climate and decreased perceived teamwork climate. From participant interviews, we identified that production pressure and time constraints were the biggest barriers to implementation and the greatest enabler was the organization's support for staff education initiatives. Most participants perceived the content of the program to be useful. These results highlight the complexities inherent in the development and evaluation of interdisciplinary patient safety interventions.

Assessment of Scrub Practitioners’ List of Intraoperative Non‐Technical Skills (SPLINTS) in an Asian Ambulatory Surgical Setting

“…These sessions included a review of direct observations recorded from an overview camera system mounted in the ORs; the researchers also could familiarize themselves with observing and assessing nonsimulated ambulatory ophthalmic procedures and validate they were able to code their observations in the same way. Gillespie recommends that new raters be tested with at least five observations, or 10% of the total sample, to improve IRR . Because both assessors in our research study have a different amount of experience and qualifications, the high IRR was encouraging.…”

Observational Tools That Quantify Nontechnical Skills in the Operating Room: A Systematic Review

“…The most frequent number of observers was two (n ¼ 15). 4,[11][12][13]23,24,[26][27][28][29][30][31]33,[36][37][38][39] Thirteen studies involved an observer with a clinical background (e.g. surgeon).…”

“…4,11,13,16,23,24,26,27,[35][36][37][38][39] Fifteen studies described the training undertaken by observers. 4,11,16,23,24,[27][28][29][30][31]33,[36][37][38][39] Validity and reliability. Validity of NOTECHS was demonstrated by 16 studies (13 of which occurred in the OR and three occurred in simulation settings).…”

“…Psychometric properties for the 31 tools are reported in Table 3 and in online supplementary material file B provides study results related to validity and reliability. Specifically, the included studies demonstrated content validity (Oxford NOTECHS), 27 predictive validity (Oxford NOTECHS, n ¼ 3; Revised NOTECHS, n ¼ 3; NOTECHS (modified), n ¼ 1; and Oxford NOTECHS II, n ¼ 4), 4,11,26,27,30,31,[33][34][35][36]38 concurrent validity (Oxford NOTECHS, n ¼ 5; Revised NOTECHS, n ¼ 2; and Oxford NOTECHS, n ¼ 1), [11][12][13]16,27,28,33,39 and face validity (Oxford NOTECHS II). 39 Ten studies assessed inter-rater reliability (Cronbach a > 0.…”

“…Across the 23 studies, the most used two observers (n ¼ 11) 17,64,86,90,91,94,95,[97][98][99]101 ; nine studies had observers with a clinical background 8,9,50,64,68,86,90,93,98 ; and eight studies had observers with a nonclinical background. 50 Non-technical skills * NOTECHS 2 24,25 Oxford non-technical skills Oxford NOTECHS 6 [11][12][13]16,26,27 Revised non-technical skills Revised NOTECHS 6 [28][29][30][31][32][33] Non-technical skills (modified) NOTECHS (modified) 1 34 Oxford Non-technical skills II Oxford NOTECHS II 6 4,[35][36][37][38][39] Non-technical skills for surgeons y NOTSS 19 9,[40][41][42][43]…”