We sought to evaluate the validity of two non-technical skills evaluation instruments, the Anaesthetists' Non-Technical Skills (ANTS) behavioural marker system and the Ottawa Global Rating Scale (GRS), to apply them to anaesthesia training. The content validity, response process, internal structure, relations with other variables and consequences were described for validity evidence. Simulated crisis management sessions were initiated during which two trained raters evaluated the performance of postgraduate first-, second-and third-year (PGY-1, PGY-2 and PGY-3) anaesthesia residents. The study included 70 participants, composed of 24 PGY-1, 24 PGY-2 and 22 PGY-3 residents. Both instruments differentiated the non-technical skills of PGY-1 from PGY-3 residents (P <0.05). Inter-rater agreement was measured using the intraclass correlation coefficient. For the ANTS instrument, the intraclass correlation coefficients for task management, team-working, situation awareness and decision-making were 0.79, 0.34, 0.81 and 0.70, respectively. For the Ottawa GRS, the intraclass correlation coefficients for overall performance, leadership, problem-solving, situation awareness, resource utilisation and communication skills were 0.86, 0.83, 0.84, 0.87, 0.80 and 0.86, respectively. The Cronbach's alpha for internal consistency of the ANTS instrument was 0.93, and was 0.96 for the Ottawa GRS. There was a high correlation between the ANTS and Ottawa GRS. The raters reported the ease of use of the Ottawa GRS compared to the ANTS. We found sufficient evidence of validity in the ANTS instrument and the Ottawa GRS for the evaluation of non-technical skills in a simulated anaesthesia setting, but the Ottawa GRS was more practical and had higher reliability.
Objective: Non-technical skills training and assessment has been implemented in anesthesia residency training program to improve quality of patient care but have not been properly assessed. We hypothesized that trainees with good knowledge correlated with good cognitive parts of non-technical skills.Methods: Seventy anesthesia residents (24 PGY-1, 24 PGY-2 and 22 PGY-3) were assessed for their knowledge by 180-item MCQs, 5 key-feature essay questions, and 18-station OSCE’s. Subsequently, a perioperative anesthesia crisis situation was set up in the simulation lab for all residents and was video recorded. Non-technical skills were assessed by 2 independent trained raters using Anesthetists’ Non-Technical Skills (ANTS) behavioral markers. The residents’ scores were calculated to find the correlation within the ANTS rating scale.Results: The mean scores of knowledge tests were 164.3 ±18.4 out of 300 [165.5 ±18.0, 154.7 ±16.3 and 173.6 ±16.4 for PGY-1, PGY-2 and PGY-3 respectively]. The mean scores of ANTS was divided into 4 categories (rating scale 1 to 4): task management 2.9 (±0.6), teamworking 3.0 (±0.5), situation awareness 2.9 (±0.8) and decision making 2.8 (±0.7). The knowledge test results moderately correlated with ANTS score in task management, situation awareness and decision making [r=0.382 (p<0.01), r=0.433 (p<0.001) and r=0.350 (p<0.01) respectively] and weakly correlated with the teamworking category (r=0.166, p=0.16).Conclusion: Resident’s scores showed moderate correlation with non-technical skills assessment results in cognitive skills. Non-technical skills are required to be trained and assessed together with knowledge to enhance the patient’s safety and outcome.
BackgroundSimulation is widely used in airway management training.ObjectivesTo show that assigning anesthesia residents’ simulation educator roles improved cognitive learning outcomes.MethodsPostgraduate second- and third-year (PGY-2 and PGY-3) anesthesia residents were randomly assigned to three groups: a teacher group (T), a hot-seat (active participant) group (H), and an observer group (O). After a train-the-trainer session, the T group prepared simulation scenarios for difficult airway management and then conducted the simulation sessions and post-session debriefing. The H group participated in the scenarios, and the O group observed the sessions. All participants attended the post-session debriefing. Evaluation was conducted at pretest, immediate posttest, and 3 months (retention test). Score differentiation and average normalized gain were calculated. Participants completed a post-simulation class survey.ResultsParticipants were 49 residents (PGY-2 = 24, PGY-3 = 25). The T group had the highest posttest score (17.06 ± 1.23); this score significantly differed from the O group (14.75 ± 2.57, P = 0.003) but not the H group (15.64 ± 1.54, P = 0.103). The average normalized gain was significantly higher in the T group than in the H and O groups (0.51 ± 0.22, 0.18 ± 0.32, and 0.17 ± 0.47, respectively; P = 0.012). Participants retained knowledge at 3 months after the session, with no significant differences among the groups. Most participants (45%) preferred to be active scenario participants, and 20% preferred to teach. Overall satisfaction was high in all groups.ConclusionThis study showed that a teaching role can be effectively applied for residents in simulation-based education on difficult airway management to support better learning outcomes.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.