Global Rating Scales and Motion Analysis Are Valid Proficiency Metrics in Virtual and Benchtop Knee Arthroscopy Simulators

Abstract

Background Work-hour restrictions and fatigue management strategies in surgical training programs continue to evolve in an effort to improve the learning environment and promote safer patient care. In response, training programs must reevaluate how various teaching modalities such as simulation can augment the development of surgical competence in trainees. For surgical simulators to be most useful, it is important to determine whether surgical proficiency can be reliably differentiated using them. To our knowledge, performance on both virtual and benchtop arthroscopy simulators has not been concurrently assessed in the same subjects. Questions/purposes (1) Do global rating scales and procedure time differentiate arthroscopic expertise in virtual and benchtop knee models? (2) Can commercially available built-in motion analysis metrics differentiate arthroscopic expertise? (3) How well are performance measures on virtual and benchtop simulators correlated? (4) Are these metrics sensitive enough to differentiate by year of training? Methods A cross-sectional study of 19 subjects (four medical students, 12 residents, and three staff) were recruited and divided into 11 novice arthroscopists (student to Postgraduate Year [PGY] 3) and eight proficient arthroscopists (PGY 4 to staff) who completed a diagnostic arthroscopy and loose-body retrieval in both virtual and benchtop knee models. Global rating scales (GRS), procedure times, and motion analysis metrics were used to evaluate performance. CI, seconds, p = 0.002). The built-in motion analysis metrics also distinguished novices from proficient arthroscopists using the self-generated virtual loose body retrieval task scores (4 ± 1 [95% CI, 3-5] versus 6 ± 1 [95% CI, 5-7], p = 0.001). GRS scores between virtual and benchtop models were very strongly correlated (q = 0.93, p \ 0.001). There was strong correlation between year of training and virtual GRS (q = 0.8, p \ 0.001) and benchtop GRS (q = 0.87, p \ 0.001) scores. Conclusions To our knowledge, this is the first study to evaluate performance on both virtual and benchtop knee simulators. We have shown that subjective GRS scores and objective motion analysis metrics and procedure time are valid measures to distinguish arthroscopic skill on both virtual and benchtop modalities. Performance on both modalities is well correlated. We believe that training on artificial models allows acquisition of skills in a safe environment. Future work should compare different modalities in the efficiency of skill acquisition, retention, and transferability to the operating room.123 Clin Orthop Relat Res (2016) 474:956-964 DOI 10.1007/s11999-015-4510-8 Clinical Orthopaedics and Related Research ® A Publication of The Association of Bone and Joint Surgeons® GRS scales. The proficient subjects completed nearly all tasks faster

Does virtual reality simulation have a role in training trauma and orthopaedic surgeons?

“…[43][44][45][46] It is in this area of transferability that supportive evidence is lacking when compared to other surgical specialties. Multiple studies have demonstrated a 'real-world' benefit from the use of Knee arthroscopic ability on a cadaver knee in a simulated intra-operative environment using the validated Global Rating Scale, 49 arthroscopic checklist, and procedural time -for both diagnostic examination and probing examination; additionally, participants were given an untrained task (partial medial meniscectomy) to assess skill transfer, which was assessed using the same metrics as above VR-trained participants outperformed both lowfidelity trained and control groups when assessed with the GRS, for diagnostic examination, probe examination, and partial medial meniscectomy; no difference was observed between arthroscopic checklist completion between the VR and lowfidelity trained groups for the diagnostic and probe examinations, although both groups outperformed the untrained controls; 31% of participants were able to complete the partial meniscectomy vs 0% in the low-fidelity and control groups; VR-trained and low-fidelity groups showed significantly lower procedural times vs controls, but were not significantly different from each other.…”

How to Build Your Simple and Cost-effective Arthroscopic Skills Simulator

“…Improvement in arthroscopic dexterity has been measured by global rating scales and motion analysis, and significant correlation between virtual-reality and bench-top knee arthroscopy simulators has been measured. 12 The more basic and simple the simulator, the easier it would be to incorporate it into everyday training. 13 Another very important aspect of training is maintaining the skills achieved with simulation or any other form of skills development.…”

A Simulation-Based Assessment System for Computer Assisted Surgical Trainer

“…Objective assessment metrics for laparoscopic surgery skills training have been proposed by several researchers (Chang et al 2016;Cotin et al 2002;Kowalewski et al 2014;Maithel et al 2006;Retrosi et al 2015;Reiley et al 2011;Oropesa et al 2014;Ritter and Scott 2007). Some proposed to design a scoring system to classify trainees' proficiency of surgical movements (Reiley et al 2011;Oropesa et al 2014).…”