Comparative assessment of three standardized robotic surgery training methods

Abstract

Objectives• To evaluate three standardized robotic surgery training methods, inanimate, virtual reality and in vivo, for their construct validity.• To explore the concept of cross-method validity, where the relative performance of each method is compared. Materials and Methods• Robotic surgical skills were prospectively assessed in 49 participating surgeons who were classified as follows: 'novice/trainee': urology residents, previous experience <30 cases (n = 38) and 'experts': faculty surgeons, previous experience Ն30 cases (n = 11).• Three standardized, validated training methods were used:(i) structured inanimate tasks; (ii) virtual reality exercises on the da Vinci Skills Simulator (Intuitive Surgical, Sunnyvale, CA, USA); and (iii) a standardized robotic surgical task in a live porcine model with performance graded by the Global Evaluative Assessment of Robotic Skills (GEARS) tool.• A Kruskal-Wallis test was used to evaluate performance differences between novices and experts (construct validity). • Spearman's correlation coefficient (r) was used to measure the association of performance across inanimate, simulation and in vivo methods (cross-method validity). Results• Novice and expert surgeons had previously performed a median (range) of 0 (0-20) and 300 (30-2000) robotic cases, respectively (P < 0.001).• Construct validity: experts consistently outperformed residents with all three methods (P < 0.001).• Cross-method validity: overall performance of inanimate tasks significantly correlated with virtual reality robotic performance (r = -0.7, P < 0.001) and in vivo robotic performance based on GEARS (r = -0.8, P < 0.0001).• Virtual reality performance and in vivo tissue performance were also found to be strongly correlated (r = 0.6, P < 0.001). Conclusions• We propose the novel concept of cross-method validity, which may provide a method of evaluating the relative value of various forms of skills education and assessment.• We externally confirmed the construct validity of each featured training tool.

Current status of robotic training in the UK – a trainees perspective

“…13,14 This may be because at present simulation focuses on generic basic skills such as hand eye coordination and therefore simulation training, though useful in the preliminary stages of learning, may not confer additional benefit in advanced surgical skill. 1,15 A more generalised non-robotic training programme was introduced to urology trainees in south London. This was found to be useful in testing both technical and non-technical skills.…”

Performance of robotic simulated skills tasks is positively associated with clinical robotic surgical performance

“…To date, the majority of robotic simulation studies have compared simulation performance to performance outside the operating room (inanimate or animate models) [1,5,9,[14][15][16]. To our knowledge, there is only one previous study (abstract only) comparing virtual reality performance (dVSS) with bladder mobilization during RARP, where the authors were unable to show a significant correlation [17].…”

“…Despite this rapid surge in use, the development and validation of training methods has failed to keep pace. Several training platforms have undergone validation testing, including inanimate tasks [1][2][3][4][5], virtual reality exercises [6][7][8][9] and ex vivo models [5,9,10]. Validation work in these studies includes demonstration of face (realism of training tool), content (usefulness as training tool), construct (ability to distinguish between different skill levels) and cross-method (correlation across training methods) validity.…”

“…These tasks consist of a set of four inanimate models, collectively called fundamental inanimate robotic skills tasks (FIRST) [5,11], and eight selected virtual reality tasks on the da Vinci Skills Simulator (dVSS) platform [8]. These simulator tasks together encompass all of the basic skills of robotic surgery, including object pick and place, hand-to-hand transfer, instrument wrist manipulation, camera control and clutching, third-arm manipulation, suturing and energy use.…”

“…1 [5,10,12]. Composed of six domains rated on a five-point Likert scale, GEARS has been shown to be a reliable assessment tool for robotic clinical performance.…”