Objective Structured Assessment of technical skill (OSATS) in the Surgical Skills and Technology Elective Program (SSTEP): Comparison of peer and expert raters

Asif, Hamza; McInnis, Carter; Dang, Frances; Ajzenberg, Henry; Wang, Peter L.; Mosa, Adam; Ko, G. T. C.; Zevin, Boris; Mann, Stephen M.; Winthrop, Andrea

doi:10.1016/j.amjsurg.2021.03.064

Cited by 18 publications

(11 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The decision to include a peer assessment has been inspired by recent literature ( 26 – 28 ), which indicates that there is good agreement between novice and expert raters and that actually expertise in performing a given procedure is not a prerequisite to assess it. However, as this is still matter of debate ( 29 ), it needs to be considered as possible further limitation of the study.…”

Section: Discussionmentioning

confidence: 99%

The “chicken-leg anastomosis”: Low-cost tissue-realistic simulation model for esophageal atresia training in pediatric surgery

Palmisani

Sezen²,

Haag³

et al. 2022

Front. Pediatr.

View full text Add to dashboard Cite

IntroductionShifting the training from the operating room (OR) to simulation models has been proven effective in enhancing patient safety and reducing the learning time to achieve competency and increase the operative efficiency. Currently the field of pediatric surgery only offers few low-cost trainers for specialized training and these feature predominantly artificial and often unrealistic tissue. The aim of this study was to develop an easy access low-cost tissue-realistic simulation model for open training of esophageal atresia and to evaluate the acceptance in trainees and junior pediatric surgeons.Materials and methodsThe model is fashioned using reconfigured chicken skin from a chicken leg. To create a model of esophageal atresia, the chicken skin is dissected off the muscle and reconfigured around a foley catheter balloon to recreate the proximal pouch and a feeding tube to recreate the distal pouch. Surrounding structures such as the tracheo-esophageal fistula and the azygos vein can be easily added, obtaining a realistic esophageal atresia (Type C) prototype. Evaluation of model construction, usage and impact on user were performed by both a self-assessment questionnaire with pre- and post-training questions as well as observer-based variables and a revised Objective Structured Assessment of Technical Skills (OSATS) score.ResultsA total of 10 participants were constructing and using the model at two different timepoints. OSATS score for overall performance was significantly higher (p = 0.005, z = −2.78) during the second observational period [median (MD): 4,95% confidence interval CI: 3.4, 5.1] compared to the first (MD: 3, 95% CI 2.4, 4.1). Self-reported boost in confidence after model usage for performing future esophageal atresia (EA) repair and bowel anastomosis (BA) in general was significantly higher (EA: U = 1, z = −2.3, p = 0.021, BA: U = 1, z = −2.41, p = 0.016) in participants with more years in training/attending status (EA MD:5, BA MD: 5.5) compared to less experienced participants (EA MD: 1.5, BA: 1).ConclusionOur easy access low-cost simulation model represents a feasible and tissue realistic training option to increase surgical performance of pediatric surgical trainees outside the OR.

show abstract

Section: Discussionmentioning

confidence: 99%

The “chicken-leg anastomosis”: Low-cost tissue-realistic simulation model for esophageal atresia training in pediatric surgery

Palmisani

Sezen²,

Haag³

et al. 2022

Front. Pediatr.

View full text Add to dashboard Cite

show abstract

“…Although peers are often novice raters, they can be taught to rate reliably using OSATS. 36 A meta-analysis found that surgical skill level does not seem to influence the validity of peer assessments, although skill level does influence self-assessments. 37 We found that when residents rated their own performance, their OSATS scores were lower than when they were scored by an expert.…”

Section: Discussionmentioning

confidence: 99%

A Study of Surgical Residents' Self-Assessment of Open Surgery Skills Using Gap Analysis

et al. 2022

View full text Add to dashboard Cite

Background: Surgical residents need structured and objective feedback to develop their skills and become capable of performing surgical procedures autonomously. A shortage of experienced surgical staff has prompted residents to seek feedback from self-assessment and peer assessments. Objectives: We investigated whether surgical residents can reliably rate their own and their peers' basic surgical skills using the Global Rating Scale (GRS) from the Objective Structured Assessment of Technical Skills. Methods: The study was a prospective and descriptive study conducted using gap analysis at the Copenhagen Academy for Medical Education and Simulation (Copenhagen, Denmark) from 2016 to 2017. Surgical residents were recruited during a course in basic open surgical skills. Among 102 course participants, 53 met the inclusion criteria and 22 participated in the study. Results: We recruited surgical residents based in the Capital Region and Zealand Region of Demark, and 42% of eligible residents participated in the study. Surgical residents underestimated their own surgical performance (median, 17 [range, 15-18] vs. 20 [range, 19.75-22]; P < 0.001). They also rated their peers lower than an experienced rater did (median 10 [range, 8.75-14] vs. 15 and median 20.5 [range, 19-22] vs. 23; both P < 0.001). Gap analysis revealed residents had unrecognized strengths (ie, self-underappraisal) in most GRS domains. Conclusions: Ratings are unreliable when surgical residents assess their own and their peers' performances using GRS. A gap analysis revealed unrecognized strengths in time and motion, instrument handling, knowledge of instruments and sutures, and knowledge of specific procedure as well as unrecognized weaknesses in flow of operation and forward planning.

show abstract

“…A pre-test on the student's knowledge on theory is completed at the beginning of the seminar (Supplemental data 1 ). This workshop includes standardized evaluation of a suture using the Objective Structured Assessment of Technical Skills (OSATS) [ 9 , 10 ] scale (Supplemental data 2 ). We incorporated video learning assessment using a comparative study with blinded evaluation.…”

Section: Methodsmentioning

confidence: 99%

Boot camp approach to surgical residency preparation: feedback from a French university hospital

et al. 2022

View full text Add to dashboard Cite

Introduction The transition from medical student to surgical resident is not a simple one. The aim of this study was to report the experience of a university hospital in the organization of the induction course for future surgical residents and the contribution of a video support in the learning of the suture. Material and method We were able to study two consecutive years of students (October 2020 and 2021). Concerning the practical and technical workshops (learning suture) we carried out a comparative study between two groups of students. A group that had video support for learning suture (video group) and a group without video (control group). The evaluation of the suture was performed in a blinded manner by two supervising surgeons. The other practical workshop was drain fixation; the students did not have a video for this workshop. A comparative study was also performed for the drain fixation workshop between the two groups (video group and control group). A program of theoretical courses was also set up. This program is established according to the different future functions of the residents by integrating medico-legal notions and teamwork. Satisfaction questionnaires were given to the students and the answers were given two months after taking up their duties in the hospital (6 questions with Likert scale and 4 free questions). Results The cohort consisted of 58 students (29 each in 2020 and 29 in 2021). Comparative analyses of the evaluation of the suture workshops showed better performance in the video group compared with the group without video. The comparison of these two groups did not show significant differences in the drain fixation workshop. The theoretical teaching was broken down according to the students' future tasks and each speaker was a specialist in his or her field of expertise. The results of the questionnaires showed a desire on the part of the students to increase the time spent on practical workshops and theoretical forensic teaching. Conclusion We were able to show through these two years of a program that we were able to offer a surgical resident preparation course. In addition, we have highlighted the contribution of a video support in the learning curve of the suture.

show abstract

Objective Structured Assessment of technical skill (OSATS) in the Surgical Skills and Technology Elective Program (SSTEP): Comparison of peer and expert raters

Cited by 18 publications

References 17 publications

The “chicken-leg anastomosis”: Low-cost tissue-realistic simulation model for esophageal atresia training in pediatric surgery

The “chicken-leg anastomosis”: Low-cost tissue-realistic simulation model for esophageal atresia training in pediatric surgery

A Study of Surgical Residents' Self-Assessment of Open Surgery Skills Using Gap Analysis

Boot camp approach to surgical residency preparation: feedback from a French university hospital

Contact Info

Product

Resources

About