Construct validity of individual and summary performance metrics associated with a computer-based laparoscopic simulator

Rivard, Justin; Vergis, Ashley; Unger, Bertram; Hardy, Krista; Andrew, Chris G; Gillman, Lawrence M.; Park, Jason

doi:10.1007/s00464-013-3414-5

Cited by 18 publications

(18 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These metrics have already been validated in previous studies [25,33,34,36]. They also showed high reliability levels with an important degree of independence to the performed tasks.…”

Section: Discussionmentioning

confidence: 55%

“…According to the results, all participants agreed with the usefulness and degree of difficulty that presented of the tasks selected for this study. In our study, the four skill tasks were chosen for two main reasons: (1) these tasks are well validated in many clinical studies [35][36][37][38][39][40][41], and (2) they contain laparoscopic skills and techniques that are usually present in many laparoscopic procedures…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Face, content, and construct validity of the EndoViS training system for objective assessment of psychomotor skills of laparoscopic surgeons

et al. 2014

View full text Add to dashboard Cite

Background The aim of this study is to present face, content, and constructs validity of the endoscopic orthogonal video system (EndoViS) training system and determines its efficiency as a training and objective assessment tool of the surgeons' psychomotor skills. Methods Thirty-five surgeons and medical students participated in this study: 11 medical students, 19 residents, and 5 experts. All participants performed four basic skill tasks using conventional laparoscopic instruments and EndoViS training system. Subsequently, participants filled out a questionnaire regarding the design, realism, overall functionality, and its capabilities to train hand-eye coordination and depth perception, rated on a 5-point Likert scale. Motion data of the instruments were obtained by means of two webcams built into a laparoscopic physical trainer. To identify the surgical instruments in the images, colored markers were placed in each instrument. Thirteen motion-related metrics were used to assess laparoscopic performance of the participants. Statistical analysis of Flores • performance was made between novice, intermediate, and expert groups. Internal consistency of all metrics was analyzed with Cronbach's a test.Results Overall scores about features of the EndoViS system were positives. Participants agreed with the usefulness of tasks and the training capacities of EndoViS system (score >4). Results presented significant differences in the execution of three skill tasks performed by participants. Seven metrics showed construct validity for assessment of performance with high consistency levels. Conclusions EndoViS training system has been successfully validated. Results showed that EndoViS was able to differentiate between participants of varying laparoscopic experience. This simulator is a useful and effective tool to objectively assess laparoscopic psychomotor skills of the surgeons. Keywords Laparoscopic surgery • Surgical training • Objective assessment • Motion metrics • Validation • Endoscopic orthogonal video system (EndoViS)Laparoscopic surgery has become an important technique within several surgical specialties, such as general surgery, gynecology, and urology. This minimally invasive technique offers many benefits for the patients as less postoperative pain, better cosmetics results, and shorter periods of hospitalization [1][2][3]. Laparoscopic surgery, however, demands additional psychomotor abilities and skills different from those in conventional open surgery [4,5].Traditionally, surgical residents acquire minimally invasive skills based on the classic apprenticeship model with hands-on training in the operation room [6,7]. This training method is not efficient, prolongs the learning curve of surgeons, and represents a potential risk to patient safety. Due to concerns for medical safety, it is essential the development of training methods for safe practice of laparoscopic surgery outside of the operating theater with the additional assessment of the surgical skills of surgeons.Currently, laparoscopic sur...

show abstract

“…These metrics have already been validated in previous studies [25,33,34,36]. They also showed high reliability levels with an important degree of independence to the performed tasks.…”

Section: Discussionmentioning

confidence: 55%

Section: Discussionmentioning

confidence: 99%

Face, content, and construct validity of the EndoViS training system for objective assessment of psychomotor skills of laparoscopic surgeons

et al. 2014

View full text Add to dashboard Cite

show abstract

“…Valid and reliable methods of assessing technical performance are essential for surgical training programs and education research . Objective assessment of technical skills of surgical trainees is an important topic in the field of surgical education.…”

Section: Introductionmentioning

confidence: 99%

Comparison of 2 training programs for basic laparoscopic skills and simulated surgery performance in veterinary students

et al. 2017

View full text Add to dashboard Cite

Objective: To compare the effects of 2 training curricula on laparoscopic skills and performance of simulated surgery in veterinary students.Study design: Prospective study.Sample population: Veterinary students (n 5 33) with no prior hands-on experience in minimally invasive surgery.Methods: Basic laparoscopic skills (BLS) were assessed based on 5 modified McGill inanimate system for training and evaluation of laparoscopic skills. Motion metrics and an objective structured assessment of technical skills (OSATS) were used to evaluate surgical skills during a simulated laparoscopic cholecystectomy performed in an augmented reality simulator. Students were randomly assigned to either skill-based (group A) or procedural-based (group B) training curriculum. Both tests were performed prior to and after a 10-session training curriculum.Results: Post-training BLS results were improved in both training groups (P < .001). Seven participants completed both presimulated and postsimulated laparoscopic cholecystectomy, preventing paired analysis. Based on motion metrics analysis, participants completed tasks in a shorter time (P 5 .0187), and with better economy of movement (P 5 .0457) after training. No difference was detected in OSATS before and after training.Conclusion: Both training curricula improved BLS, but significant differences were not detected between the procedural-based training program and basic skills training alone in veterinary students. Motion metrics such as time, economy of movement, and instrument path were superior to an OSATS, when assessing surgical performance. Further studies are needed to compare the effects of different simulators on the training of veterinarians with diverse laparoscopic surgical experience. | I NT ROD UCTI ONMinimal invasive surgery (MIS) results in less pain and faster recovery for animal patients 1-5 but requires different surgical skills than traditional open surgery. 6,7 Surgeons need to overcome the fulcrum effect, adjust hand-eye coordination, altering the approaching angle of instruments, and adjust to a more restricted field of view. 8 The surgical mantra "see one, do one, teach one" is not suitable in laparoscopic surgery. 9 In the past 2 decades, interest in simulation training and assessment has grown, to surmount the training obstacle in MIS, in animals and man. [8][9][10][11][12][13][14] Simulation aims to replicate clinical situations, ranging from low fidelity simulated basic skills training, to very realistic surgical simulations in high fidelity models. Accordingly, health educators in human medicine have adopted simulation as a viable educational method to teach and practice a diverse range of clinical skills. Live animal or cadaver models are high fidelity and useful training and assessment tools, 13,28 but are sparsely available, very expensive, and the ethical concern of training in live animals remains an important aspect. A recent study showed the value of motion metrics in evaluation of laparoscopic skills in veterinary medicine, 29 yet the effe...

show abstract

“…1 Having valid and reliable methods of assessing technical performance is essential for surgical training programs and education research. 2 This helps to maintain academic standards, track progress, and may function as a method to provide feedback to trainees on both their strengths and weaknesses. Despite its importance in surgical training, formal objective assessment of technical ability and competence has lagged behind other educational endeavors in surgical training programs.…”

mentioning

confidence: 99%

Establishing the concurrent validity of general and technique-specific skills assessments in surgical education

Steigerwald

Park

Hardy

et al. 2016

The American Journal of Surgery

Self Cite

View full text Add to dashboard Cite

Construct validity of individual and summary performance metrics associated with a computer-based laparoscopic simulator

Cited by 18 publications

References 38 publications

Face, content, and construct validity of the EndoViS training system for objective assessment of psychomotor skills of laparoscopic surgeons

Face, content, and construct validity of the EndoViS training system for objective assessment of psychomotor skills of laparoscopic surgeons

Comparison of 2 training programs for basic laparoscopic skills and simulated surgery performance in veterinary students

Establishing the concurrent validity of general and technique-specific skills assessments in surgical education

Contact Info

Product

Resources

About