Observational Tools That Quantify Nontechnical Skills in the Operating Room: A Systematic Review

McMullan, Ryan; Urwin, Rachel; Sunderland, Neroli; Westbrook, Johanna I.

doi:10.1016/j.jss.2019.10.012

Cited by 15 publications

(13 citation statements)

References 104 publications

(255 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Two researchers independently reviewed the titles and abstracts against the inclusion and exclusion criteria. One researcher (JC) reviewed all articles twice, and another (DY) reviewed a random 10% of articles following previous published protocol (McMullan et al, 2020). Full-text articles were downloaded and assessed for eligibility if the inclusion of an article could not be determined from the initial screening.…”

Section: Methodsmentioning

confidence: 99%

“…Centered around NTS constructs, many role- and specialty-specific assessment tools have been developed to evaluate NTS behavior. A recent review identified 31 observational NTS tools for the OR (McMullan et al, 2020), which included evaluation ranging from that of the entire team (e.g., Observational Teamwork Assessment for Surgery [OTAS]) down to the individual surgical team member such as the attending surgeon (e.g., Nontechnical Skills for Surgeons [NOTSS] and Oxford Nontechnical Skills [NOTECHS]), anesthesiologist (e.g., Anesthetists’ Nontechnical Skills [ANTS]), and scrub nurse (Scrub Practitioner’s List of Intraoperative Nontechnical Skills [SPLINTS]; Fletcher et al, 2003; Mitchell et al, 2012; Robertson et al, 2014; Undre et al, 2007; Yule et al, 2008). These behavior rating systems have been used for NTS assessment primarily through expert observation in the OR.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Objective Measures of Surgeon Non-Technical Skills in Surgery: A Scoping Review

Cha

2021

Hum Factors

View full text Add to dashboard Cite

Objective The purpose of this study was to identify, synthesize, and discuss objective behavioral or physiological metrics of surgeons’ nontechnical skills (NTS) in the literature. Background NTS, or interpersonal or cognitive skills, have been identified to contribute to safe and efficient surgical performance; however, current assessments are subjective, checklist-based tools. Intraoperative skill evaluation, such as technical skills, has been previously utilized as an objective measure to address such limitations. Methods Five databases in engineering, behavioral science, and medicine were searched following PRISMA reporting guidelines. Eligibility criteria included studies with NTS objective measurements, surgeons, and took place within simulated or live operations. Results Twenty-three articles were included in this review. Objective metrics included communication metrics and measures from physiological responses such as changes in brain activation and motion of the eye. Frequencies of content-coded communication in surgery were utilized in 16 studies and were associated with not only the communication construct but also cognitive constructs of situation awareness and decision making. This indicates the underlying importance of communication in evaluating the NTS constructs. To synthesize the scoped literature, a framework based on the one-way communication model was used to map the objective measures to NTS constructs. Conclusion Objective NTS measurement of surgeons is still preliminary, and future work on leveraging objective metrics in parallel with current assessment tools is needed. Application Findings from this work identify objective NTS metrics for measurement applications in a surgical environment.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Objective Measures of Surgeon Non-Technical Skills in Surgery: A Scoping Review

Cha

2021

Hum Factors

View full text Add to dashboard Cite

show abstract

“…might suspect that different scores could have been observed with different scales. 40,41 One might acknowledge that even if the two nontechnical skills scales are somewhat different, they are partly composed of elements that could be measuring the same behavior and would be "double counted" in their presence or absence. This choice may have influenced the overall performance score.…”

Section: Educationmentioning

confidence: 99%

Stress Management Training Improves Overall Performance during Critical Simulated Situations

et al. 2020

View full text Add to dashboard Cite

Background High-fidelity simulation improves participant learning through immersive participation in a stressful situation. Stress management training might help participants to improve performance. The hypothesis of this work was that Tactics to Optimize the Potential, a stress management program, could improve resident performance during simulation. Methods Residents participating in high-fidelity simulation were randomized into two parallel arms (Tactics to Optimize the Potential or control) and actively participated in one scenario. Only residents from the Tactics to Optimize the Potential group received specific training a few weeks before simulation and a 5-min reactivation just before beginning the scenario. The primary endpoint was the overall performance during simulation measured as a composite score (from 0 to 100) combining a specific clinical score with two nontechnical scores (the Ottawa Global Rating Scale and the Team Emergency Assessment Measure scores) rated for each resident by four blinded independent investigators. Secondary endpoints included stress level, as assessed by the Visual Analogue Scale during simulation. Results Of the 134 residents randomized, 128 were included in the analysis. The overall performance (mean ± SD) was higher in the Tactics to Optimize the Potential group (59 ± 10) as compared with controls ([54 ± 10], difference, 5 [95% CI, 1 to 9]; P = 0.010; effect size, 0.50 [95% CI, 0.16 to 0.91]). After specific preparation, the median Visual Analogue Scale was 17% lower in the Tactics to Optimize the Potential group (52 [42 to 64]) than in the control group (63 [50 to 73]; difference, −10 [95% CI, −16 to −3]; P = 0.005; effect size, 0.44 [95% CI, 0.26 to 0.59]. Conclusions Residents coping with simulated critical situations who have been trained with Tactics to Optimize the Potential showed better overall performance and a decrease in stress level during high-fidelity simulation. The benefits of this stress management training may be explored in actual clinical settings, where a 5-min Tactics to Optimize the Potential reactivation is feasible prior to delivering a specific intervention. Editor’s Perspective What We Already Know about This Topic What This Article Tells Us That Is New

show abstract

“…The result of the consensus process is a validated set of RAS-critical behaviours. It comprises teamwork behaviours that are assessable, tangible and can be utilized for training, simulation, and teamwork improvement measures [13,[49][50][51][52].…”

Section: Discussionmentioning

confidence: 99%

RAS-NOTECHS: validity and reliability of a tool for measuring non-technical skills in robotic-assisted surgery settings

et al. 2021

View full text Add to dashboard Cite

Background Non-technical skills (NTS) are essential for safe surgical practice as they impact workflow and patient outcomes. Observational tools to measure operating room (OR) teams’ NTS have been introduced. However, there are none that account for the specific teamwork challenges introduced by robotic-assisted surgery (RAS). We set out to develop and content-validate a tool to assess multidisciplinary NTS in RAS. Methodology Stepwise, multi-method procedure. Observations in different surgical departments and a scoping literature review were first used to compile a set of RAS-specific teamwork behaviours. This list was refined and expert validated using a Delphi consensus approach consisting of qualitative interviews and a quantitative survey. Then, RAS-specific behaviours were merged with a well-established assessment tool on OR teamwork (NOTECHS II). Finally, the new tool—RAS-NOTECHS—was applied in standardized observations of real-world procedures to test its reliability (inter-rater agreement via intra-class correlations). Results Our scoping review revealed 5242 articles, of which 21 were included based on pre-established inclusion criteria. We elicited 16 RAS-specific behaviours from the literature base. These were synthesized with further 18 behavioural markers (obtained from 12 OR-observations) into a list of 26 behavioural markers. This list was reviewed by seven RAS experts and condensed to 15 expert-validated RAS-specific behavioural markers which were then merged into NOTECHS II. For five observations of urologic RAS procedures (duration: 13 h and 41 min), inter-rater agreement for identification of behavioural markers was strong. Agreement of RAS-NOTECHS scores indicated moderate to strong agreement. Conclusions RAS-NOTECHS is the first observational tool for multidisciplinary NTS in RAS. In preliminary application, it has been shown to be reliable. Since RAS is rapidly increasing and challenges for effective and safe teamwork remain at the forefront of quality and safety of surgical care, RAS-NOTECHS may contribute to training and improvement efforts in technology-facilitated surgeries.

show abstract

Observational Tools That Quantify Nontechnical Skills in the Operating Room: A Systematic Review

Cited by 15 publications

References 104 publications

Objective Measures of Surgeon Non-Technical Skills in Surgery: A Scoping Review

Objective Measures of Surgeon Non-Technical Skills in Surgery: A Scoping Review

Stress Management Training Improves Overall Performance during Critical Simulated Situations

RAS-NOTECHS: validity and reliability of a tool for measuring non-technical skills in robotic-assisted surgery settings

Contact Info

Product

Resources

About