Objective assessment in residency‐based training for transoral robotic surgery

Abstract

Objective To develop a robotic surgery training regimen integrating objective skill assessment for otolaryngology and head and neck surgery trainees consisting of training modules of increasing complexity and leading up to procedure specific training. In particular, we investigate applications of such a training approach for surgical extirpation of oropharyngeal tumors via a transoral approach using the da Vinci Robotic system. Study Design Prospective blinded data collection and objective evaluation (OSATS) of three distinct phases using the da Vinci Robotic surgical system. Setting Academic University Medical Engineering/Computer Science laboratory Methods Between September 2010 and July 2011, 8 Otolaryngology Head and Neck Surgery residents and 4 staff “experts” from an academic hospital participated in three distinct phases of robotic surgery training involving 1) robotic platform operational skills, 2) set-up of the patient side system, and 3) a complete ex-vivo surgical extirpation of an oropharyngeal “tumor” located in the base of tongue. Trainees performed multiple (4) approximately equally spaced training sessions in each stage of the training. In addition to trainees, baseline performance data was obtained for the experts. Each surgical stage was documented with motion and event data captured from the application programming interfaces (API) of the da Vinci system, as well as separate video cameras as appropriate. All data was assessed using automated skill measures of task efficiency, and correlated with structured assessment (OSATS, and similar Likert scale) from three experts to assess expert and trainee differences, and compute automated and expert assessed learning curves. Results Our data shows that such training results in an improved didactic robotic knowledge base and improved clinical efficiency with respect to the set-up and console manipulation. Experts (e.g. average OSATS 25, Stdev. 3.1, module 1 – suturing) and trainees (average OSATS 15.9, Stdev. 3.9, week 1) are well separated at the beginning of the training, and the separation reduces significantly (expert average OSATS 27.6, Std. 2.7, trainee average OSATS 24.2, Std. 6.8, module 3) at the conclusion of the training. Learning curves in each of the three stages show diminishing differences between the experts and trainees, also consistent with expert assessment. Subjective assessment by experts verified the clinical utility of the module 3 surgical environment and a survey of trainees consistently rated the curriculum as very useful in progression to human operating room assistance. Conclusions Structured curricular robotic surgery training with objective assessment promises to reduce the overhead for mentors, allow detailed assessment of human-machine interface skills and create customized training models for individualized training. This preliminary study verifies the utility of such training in improving human-machine operations skills (module 1), and operating room and surgical skills (module 2 and 3). In contrast to cur...

The University of Pennsylvania Curriculum for Training Otorhinolaryngology Residents in Transoral Robotic Surgery

“…The training program we introduced includes the common elements of didactics, skills training, and operative experience, which we found described in structured training curricula for other specialties. While we found two other descriptions of structured modules for training residents in the field of head and neck surgery, notably these modules focused solely on training exercises in an inanimate robotics laboratory setting [11,12]. The structured curriculum described in this report differs by additionally focusing on training in didactics and operative modules.…”

“…While no two curricula were exactly the same, common elements across the many programs could be identified. In otorhinolaryngology, there were two prior descriptions of structured modules, based solely in a robotic skills training laboratory [11,12]. However, in the other curricula, important structured elements include didactics teaching, skills practice in laboratory settings, and progressive surgical skills development in the operating room.…”

The Generic Error Rating Tool: A Novel Approach to Assessment of Performance and Surgical Education in Gynecologic Laparoscopy

“…24 We used a cutoff of 28 on the OSATS scale to group surgeons, because this represents 80% of the maximal OSATS score, and previous studies in other specialties have shown that trained staff surgeons generally scored at least 80% on global rating scales. 25,26 Furthermore, this cutoff has previously been used as a competency benchmark in laparoscopic hysterectomies. 27 In addition to analyzing the entire procedures, we also analyzed operative substeps to assess whether GERT analysis allows the detection of procedure steps that are more prone to technical errors.…”

Toward intraoperative image-guided transoral robotic surgery

“…A future study will include Otolaryngology-Head and Neck residents training in TORS to determine the effectiveness of the system for non-expert / novice surgeons. Currently, methods of training residents on the daVinci for TORS [16] typically involve resection of an embedded sphere from tongues of porcine animal phantoms, and results are evaluated by measuring margins surrounding the target from the resected sample. Overlay of the tumor in addition to subsurface information would assist in achieving adequate negative margins, crucial to prevent recurrence of disease in clinical practice.…”