In Response To objective assessment in residency‐based training for transoral robotic surgery

Abstract

In Reply:We thank the readers for their careful consideration and review of our study and are encouraged by their sincere feedback.To update the readers, we are now preparing the next iteration of a curricular resident training protocol and have expanded the training beyond the base of tongue resection already published. We expect to provide the community with additional publications and updates when the next iteration is complete and results are available. In addition, our methods (and even equipment and resources where possible) are now available to the research community. Our work has also been reviewed by the Minimally Invasive Robotic Association Fundamentals of Robotic Surgery workgroup, and we have provided didactic input to that effort, which might someday be the accepted credentialing platform across specialties. In addition, we continue to mine our data for additional insights.With respect to the specific suggestions made by the readers, they are good advice for any research study. However, the presented study was well controlled for the effects of additional instruction. No instruction for the specific training tasks was provided during the training modules, and no additional disclosures are needed. We do agree that reviewing the design for needed controls and sources of bias is sound advice for any research study.We disagree with the interpretation and commentary on interobserver agreement. Our reviewers were well trained with pregraded example studies, and the rater training process was sound. The 0.8 "good" level (noted in the readers' letter) finds little support in broader analysis, and we used broadly accepted levels to classify our variability results. The levels measured are suitable for instructional use. We further note that these statistics (will) continue to evolve with data from additional protocols (both users and raters), and we will report such updates when available. It was not an aim of the study to improve rater agreement, we merely reported the levels achieved in this iteration of the protocol. It must be emphasized that the order of detail and complexity of our measurements and analysis is orders of magnitude greater than previous reports, and a comparison may be moot to begin with.We note that learning curves reported are for an individual trainee over the considerable duration of the protocol and are compared with expert performance. The additional statistical analysis recommended, although relevant, deviates from the primary goals of the first iteration of our protocol. We aimed to establish performance baselines and verify the efficacy of measurement and analysis methods, and that expert baselines were achievable for our experimental tasks given the various constraints (including time available) for today's trainees.In summary, we again thank the writers for their feedback. We will incorporate additional statistics, over additional raters and trainees, and over the following iterations of our protocol in follow-up publications. We also encourage this group, and other researcher...