Multi‐institutional validation of a perfused robot‐assisted partial nephrectomy procedural simulation platform utilizing clinically relevant objective metrics of simulators (CROMS)

Abstract

Objective To conduct a multi‐institutional validation of a high‐fidelity, perfused, inanimate, simulation platform for robot‐assisted partial nephrectomy (RAPN) using incorporated clinically relevant objective metrics of simulation (CROMS), applying modern validity standards. Materials and Methods Using a combination of three‐dimensional (3D) printing and hydrogel casting, a RAPN model was developed from the computed tomography scan of a patient with a 4.2‐cm, upper‐pole renal tumour (RENAL nephrometry score 7×). 3D‐printed casts designed from the patient’s imaging were used to fabricate and register hydrogel (polyvinyl alcohol) components of the kidney, including the vascular and pelvicalyceal systems. After mechanical and anatomical verification of the kidney phantom, it was surrounded by other relevant hydrogel organs and placed in a laparoscopic trainer. Twenty‐seven novice and 16 expert urologists, categorized according to caseload, from five academic institutions completed the simulation. Results Clinically relevant objective metrics of simulators, operative complications, and objective performance ratings (Global Evaluative Assessment of Robotic Skills [GEARS]) were compared between groups using Wilcoxon rank‐sum (continuous variables) and parametric chi‐squared (categorical variables) tests. Pearson and point‐biserial correlation coefficients were used to correlate GEARS scores to each CROMS variable. Post‐simulation questionnaires were used to obtain subjective supplementation of realism ratings and training effectiveness. Results Expert ratings demonstrated the model’s superiority to other procedural simulations in replicating procedural steps, bleeding, tissue texture and appearance. A significant difference between groups was demonstrated in CROMS [console time (P < 0.001), warm ischaemia time (P < 0.001), estimated blood loss (P < 0.001)] and GEARS (P < 0.001). Six major intra‐operative complications occurred only in novice simulations. GEARS scores highly correlated with the CROMS. Conclusions This perfused, procedural model offers an unprecedented realistic simulation platform, which incorporates objective, clinically relevant and procedure‐specific performance metrics.

What are clinically relevant performance metrics in robotic surgery? A systematic review of the literature

“…Together, their description of metrics included CRPMs, CROMs, PBP metrics, and reference-procedure metrics. Countries of publication included Germany [11,12], England [13][14][15][16][17], and the USA [6,18].…”

“…Witthaus et al, introduced CRPMs as "concepts to design a conceptual framework for incorporating measures pertinent to a surgical task within a high-fidelity procedural simulation construct" [17]. Ghazi et al, defined CROMs as measures that "extend beyond basic robotic skills training into procedure-specific training" and provide tailored feedback to allow surgeons to progress based on individualised capabilities [15]. Ma et al, stated that CRPMs were those utilized to provide procedure-tailored feedback for surgical training and therefore "expedite the acquisition of robotic suturing skills" for each individual surgeon [18].…”

“…This enabled the incorporation of tailored clinically relevant performance metrics in training for nerve-sparing robot-assisted radical prostatectomy (NS-RARP) and Robot-assisted partial nephrectomy (RAPN). The metrics included: applied force to the neurovascular bundle during dissection, post-simulation margin status, UVA integrity, task-specific operating tasks, estimated blood loss [17] as well as console time, warm ischemia time (WIT), and positive surgical margins (PSMs) [15], respectively. Methodology for developing clinically relevant metrics for UVA utilised pre-existing validated metrics including APMs and RACE score [18].…”

“…CRPMs for NS-RARP were content validated by performing nerve sensor calibration, surgical margin verification and using the standard 180 ml UVA leak test [17]. An iterative development process was used to assess feedback and the feasibility of the CROMs in relation to the RAPN [15]. APMs related to UVA steps were collated from data from the Da Vinci robotic system, and combined with technical skill scores from RACE, which was previously validated [18].…”

Articles on robotic surgery

“…Use of simulation training for registrars will allow skill acquisition by the trainee on their learning curve before patient exposure on the operating table. This education system would use a combination of online learning, simulation training similar to aviation education, virtual reality lessons, surgery on simple models and then surgery on high‐fidelity human organ models made from polyvinyl alcohol 14 . Appropriately, the surgeon then progresses to mentored live human surgery recorded on video and scored by the surgeon trainer.…”