Consensus guidelines for validation of virtual reality surgical simulators

Abstract

The Work Group for Evaluation and Implementation of Simulators and Skills Training Programmes is a newly formed sub-group of the European Association of Endoscopic Surgeons (EAES). This work group undertook a review of validation evidence for surgical simulators and the resulting consensus is presented in this article. Using clinical guidelines criteria, the evidence for validation for six different simulators was rated and subsequently translated to a level of recommendation for each system. The simulators could be divided into two basic types; systems for laparoscopic general surgery and flexible gastrointestinal endoscopy. Selection of simulators for inclusion in this consensus was based on their availability and relatively widespread usage as of July 2004. Whilst level 2 recommendations were achieved for a few systems, it was clear that there was an overall lack of published validation studies with rigorous experimental methodology. Since the consensus meeting, there have been a number of new articles, system upgrades and new devices available. The work group intends to update these consensus guidelines on a regular basis, with the resulting article available on the EAES website (http://www.eaes-eur.org ).

La competencia digital docente: análisis de la autopercepción y evaluación del desempeño de los estudiantes universitarios de educación por medio de un entorno 3D

“…Según Carter et al (2005), la evaluación debe ser capaz de demostrar varias formas de validez. El nivel más básico es la validez aparente o de apariencia, el grado de semejanza entre el entorno y la actividad real (Schreuder, van Dongen, Roeleveld, Schijven, Broeders, 2009).…”

“…La validez de constructo es el grado en que una prueba mide el rasgo que pretende medir, por la que es posible, por ejemplo, discriminar entre los diferentes niveles de capacidad o experiencia, y la validez de contenido es el grado en que la competencia que se quiere evaluar es medida a través de la herramienta (Coles, 2011). Existen otras medidas de validez, como la validez de criterio, la concurrente o la validez predictiva, que también pueden aportar información útil en este proceso (Carter et al 2005;Coles, 2011). Sin embargo, la validez en experiencias de evaluación formativa, en este tipo de pruebas de desempeño y en entornos virtuales 3D, puede…”

Training basic laparoscopic skills using a custom-made video game

“…Although Underground is not an actual simulator, it will be validated through a traditional validation process, as described by the European Association of Endoscopic Surgeons in their consensus guidelines for the validation of virtual reality surgical simulators [8]. In this process, called test validation, face, content, construct and concurrent validity of a simulator need to be obtained before one can state that such a machine actually measures or tests what it is supposed to: basic laparoscopic skills.…”

Operator experience determines performance in a simulated computer-based brain tumor resection task

“…More specifically, operating in the VR environment of the simulator must feel like operating on a patient in physical reality (face validity), the simulator must consistently record data on important aspects of surgical performance (content validity), the measurements made by the simulator must be accurate and the intended variables (construct validity), and data obtained from the simulator must correlate with data collected with other validated measures of surgical performance [8]. In terms of reliability, the user experience must be consistent each time and, without a change in surgical skill, performance on the simulator should not change.…”