Verification of accurate technical insight: a prerequisite for self-directed surgical training

Abstract

BACKGROUND Simulation-based surgical skills training during preclinical education is a persistent challenge due to time constraints of trainees and instructors alike. Self-directed practice is resource-efficient and flexible; however, insight into technical proficiency among trainees is often lacking. The purpose of this study is to prospectively assess the accuracy of self-assessments among medical students learning basic surgical suturing. METHODS Over seven weekly practice sessions, preclinical medical students performed serial repetitions of a simulation-based suturing task under one-on-one observation by one of four trainers. Following each task repetition, self- and trainer-assessments were performed using a 36-point weighted checklist of technical standards developed a priori by expert consensus. Upon study completion, agreement between self- and trainer-assessments was measured using weighted Cohen’s kappa coefficients. RESULTS Twenty-nine medical students each performed a median of 25 suture task repetitions (IQR 21.5–28). Self-assessments tended to overestimate proficiency during the first tertile of practice attempts. Agreement between self- and trainer-assessments improved with experience, such that the weighted kappa statistics for the two-handed and instrument ties were greater than 0.81 after 18 to 21 task attempts. CONCLUSIONS Inexperienced trainees frequently overestimate technical proficiency through self-assessments. However, this bias diminishes with repetitive practice. Only after trainees have attained the capacity to accurately self-assess can effective self-directed learning take place.

Are Self-Assessment and Peer Assessment of Added Value in Training Complex Pediatric Surgical Skills?

“…This is in line with the study of Hu et al who stated that the correlation between self-and expert assessment will increase when the experience of the participants increases. 11 The higher overall performance score of the second group suggests that reflection-before-practice does improve the overall performance of the trainee, as Ganni et al had suggested before. 10 They argue that selfassessment can be used as a means of improving reflective practice and subsequently improve the trainee's overall professional competence and skills, rather than trying to achieve accordance between expert and self-assessment and a reduction in overestimation of performance.…”

“…9,10 Hu et al stated that the correlation between self-assessment and expert-assessment improved with the gain of skills of students, although they studied this with medical students and not surgical residents, who already have basic skills. 11 However, others state that peer assessment is superior to self-assessment because there is less overscoring by peers and this could make it less stressful for the trainee. 12 In the field of pediatric surgery, assessment is of uttermost importance.…”

Development of a Web Application That Evaluates Suture Performance in Off-the-Job Training

“…Third, click the Trace button (2) and trace the stitch line. If you fail to trace the line, use the Erase or Clear button (3) and (4). Finally, click the Upload button, and the score will be displayed on the right side (Fig.…”

“…However, most reports examining training systems use the subjective scores assigned by supervising surgeons. [1][2][3][4] Time taken for procedures has been preferably used in literatures as a gold standard of objective evaluation. [5][6][7] Although a few studies adopted the thread tension as the objective evaluating factor, 8,9) it cannot necessarily demonstrate the whole procedural achievement.…”

A pre-clerkship simulation-based procedural skills curriculum: decreasing anxiety and improving confidence of procedural skill performance

“…Moreover, longitudinal training and spaced repetition have shown to contribute to a high degree of procedural competence over time. [13][14][15][16] A flipped classroom approach is also a beneficial addition to a training program as it significantly improves learner performance when compared to classroom-based teaching. 17 Finally, procedural skill instruction by senior medical students, known as a near-peer model, is as effective for procedural skill acquisition as the instruction offered by staff physicians, 13 while also developing the educational ability of student teachers.…”