Development and validation of the Skills Assessment in Microsurgery for Brain Aneurysms (SAMBA) instrument for predicting proficiency in aneurysm surgery

Abstract

<jats:sec><jats:title>OBJECTIVE</jats:title><jats:p>Surgical performance evaluation was first described with the OSATS (Objective Structured Assessment of Technical Skills) and modified for aneurysm microsurgery simulation with the OSAACS (Objective Structured Assessment of Aneurysm Clipping Skills). These methods rely on the subjective opinions of evaluators, however, and there is a lack of objective evaluation for proficiency in the microsurgical treatment of brain aneurysms. The authors present a new instrument, the Skill Assessment in Microsurgery for Brain Aneurysms (SAMBA) scale, which can be used similarly in a simulation model and in the treatment of unruptured middle cerebral artery (MCA) aneurysms to predict surgical performance; the authors also report on its validation.</jats:p></jats:sec><jats:sec><jats:title>METHODS</jats:title><jats:p>The SAMBA scale was created by consensus among 5 vascular neurosurgeons from 2 different neurosurgical departments. SAMBA results were analyzed using descriptive statistics, Cronbach’s alpha indexes, and multivariate ANOVA analyses (p &lt; 0.05).</jats:p></jats:sec><jats:sec><jats:title>RESULTS</jats:title><jats:p>Expert, intermediate-level, and novice surgeons scored, respectively, an average of 33.9, 27.1, and 16.4 points in the real surgery and 33.3, 27.3, and 19.4 points in the simulation. The SAMBA interrater reliability index was 0.995 for the real surgery and 0.996 for the simulated surgery; the intrarater reliability was 0.983 (Cronbach’s alpha). In both the simulation and the real surgery settings, the average scores achieved by members of each group (expert, intermediate level, and novice) were significantly different (p &lt; 0.001). Scores among novice surgeons were more diverse (coefficient of variation = 12.4).</jats:p></jats:sec><jats:sec><jats:title>CONCLUSIONS</jats:title><jats:p>Predictive validation of the placenta brain aneurysm model has been previously reported, but the SAMBA scale adds an objective scoring system to verify microsurgical ability in this complex operation, stratifying proficiency by points. The SAMBA scale can be used as an interface between learning and practicing, as it can be applied in a safe and controlled environment, such as is provided by a placenta model, with similar results obtained in real surgery, predicting real surgical performance.</jats:p></jats:sec>

Training Performance Assessment for Intracranial Aneurysm Clipping Surgery Using a Patient-Specific Mixed-Reality Simulator: A Learning Curve Study

“…Second, data were collected over a relatively short study duration (4 months). Third, although the performance indicators analyzed in this study have been extracted from existing performance scales specific to IA clipping, 22,26 none of these scales have been used in their completely validated form because of incompatibilities with the current scope of the training models (no craniotomy and durotomy, no dissection, etc.) and to reduce the review time, which was known to be a time-consuming task.…”

“…7 However, the availability of such simulators on the market is limited. While many previous studies have focused on face, content, and predictive validity, [21][22][23][24][25][26] more research is needed to investigate their impact on the learning curve for IA treatment.…”

Microsurgical education in Greece: past, present, and future

“…The development of assessment tools for Robot-Assisted Microsurgery (RAMS) skills is still in progress [26][27][28][29]. All of these innovative grading tools are comprehensive and reliable for assessing the students' progress throughout a microsurgical course [30,31]. However, they focus solely on the technical aspects [32], such as manual dexterity, hand-eye coordination, meticulous suture placement [33,34], speed, operative flow, motion 35, and patency of the anastomosis based on taskspecific checklists [36][37][38][39][40].…”

Superiority of living animal models in microsurgical training: beyond technical expertise

“…The development of assessment tools for robot-assisted microsurgery (RAMS) skills is still in progress [12][13][14][15]. All of these innovative grading tools are comprehensive and reliable for assessing the students' progress throughout a microsurgical course [16,17]. However, they focus solely on the technical aspects [18], such as manual dexterity, hand-eye coordination, meticulous suture placement [19,20], speed, operative flow, motion [21], and patency of the anastomosis based on task-specific checklists [22][23][24][25][26].…”