Utilizing a Novel Tool for the Comprehensive Assessment of Resident Operative Performance

Glarner, Carly E.; McDonald, Robert J.; Smith, Amy; Leverson, Glen; Peyré, Sarah E.; Pugh, Carla M.; Greenberg, Caprice C.; Greenberg, Jacob A.; Foley, Eugene F.

doi:10.1016/j.jsurg.2013.07.009

Cited by 42 publications

(34 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, should the WLE simulator and this system of performance scoring be adopted for trainee assessment, it would be recommended to restrict performance scoring to the steps of exposure, flap development, glandular remodelling, skin closure and overall procedural quality, as only these steps differentiated performance between all three experience groups. In general, the results add to a large body of data regarding the validity and reliability of expert ratings of trainee performance using retrospective video analysis. Importantly, the present study demonstrates not only that operators can be distinguished based on performance scoring, but also that independent and blinded assessments have excellent reliability and interobserver agreement.…”

Section: Discussionsupporting

confidence: 61%

Validation of an oncoplastic breast simulator for assessment of technical skills in wide local excision

Leff

Petrou

Mavroveli

et al. 2016

Journal of British Surgery

View full text Add to dashboard Cite

show abstract

Section: Discussionsupporting

confidence: 61%

Validation of an oncoplastic breast simulator for assessment of technical skills in wide local excision

Leff

Petrou

Mavroveli

et al. 2016

Journal of British Surgery

View full text Add to dashboard Cite

show abstract

“…These include the Operative Performance Rating System (OPRS) from Southern Illinois University, which has been validated as a more comprehensive and shorter, single-item assessment, as well as a comprehensive assessment tool from the University of Wisconsin that evaluates procedure specific, general technical, and non-technical operative skills (25, 26, 27). Other studies have attempted investigating different methods of debriefs, such as the Procedure Feedback Form that incorporates self-assessment by the resident and evaluation by staff and a study from the Mayo Clinic integrating both post-operative and team feedback prior to closing the patient (28, 29).…”

Section: Discussionmentioning

confidence: 99%

Integrating Postoperative Feedback Into Workflow: Perceived Practices and Barriers

Nathwani

Glarner

Law

et al. 2017

Journal of Surgical Education

Self Cite

View full text Add to dashboard Cite

Objective Previous studies have found that both resident and staff surgeons highly value post-operative feedback; and that such feedback has high educational value. However, little is known about how to consistently deliver this feedback. Our aim was to understand how often surgical residents should receive feedback and what barriers are preventing this from occurring. Design Surveys were distributed to residents and attending surgeons. Questions focused on the current frequency of post-operative feedback, desired frequency and methods of feedback, and perceived barriers. Quantitative data were analyzed with descriptive statistics, and text responses were examined using coding. Setting University-based general surgery department at a Midwestern institution Participants General surgery residents (n=23) and attending surgeons (n=22) participated in this study. Results Residents reported receiving and staff reported giving feedback for procedure specific performance after 25% versus 34% of cases, general technical feedback after 36% versus 32%, and non-technical performance after 17% versus 18%. Both perceived procedure-specific and general technical feedback should be given over 80% of the time, and non-technical feedback should happen for nearly 60% of cases. Verbal feedback immediately after the operation was rated as best practice. Both parties identified time, conflicting responsibilities, lack of privacy, and discomfort with giving and receiving meaningful feedback as barriers. Conclusions Both resident and staff surgeons agree that post-operative feedback is given far less often than it should. Future work should study intraoperative and post-operative feedback to validate resident and attending perceptions such that interventions to improve and facilitate this process can be developed.

show abstract

“…Methods of evaluation of resident performance in the operating room, how skills develop over the course of training, and how to best quantify operative skills remain highly variable. 8 While competency-based resident education has been previously suggested as a means of improving neurosurgical resident training, the optimal means of evaluating competency have not been established. 11 Beginning in July 2013, the Neurological Surgery Milestone Project was launched as a joint initiative between the Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Neurological Surgery (ABNS).…”

mentioning

confidence: 99%

Use of a formal assessment instrument for evaluation of resident operative skills in pediatric neurosurgery

Hadley

Lam

Briceño

et al. 2015

PED

View full text Add to dashboard Cite

OBJECT Currently there is no standardized tool for assessment of neurosurgical resident performance in the operating room. In light of enhanced requirements issued by the Accreditation Council for Graduate Medical Education’s Milestone Project and the Matrix Curriculum Project from the Society of Neurological Surgeons, the implementation of such a tool seems essential for objective evaluation of resident competence. Beyond compliance with governing body guidelines, objective assessment tools may be useful to direct early intervention for trainees performing below the level of their peers so that they may be given more hands-on teaching, while strong residents can be encouraged by faculty members to progress to conducting operations more independently with passive supervision. The aims of this study were to implement a validated assessment tool for evaluation of operative skills in pediatric neurosurgery and determine its feasibility and reliability. METHODS All neurosurgery residents completing their pediatric rotation over a 6-month period from January 1, 2014, to June 30, 2014, at the authors’ institution were enrolled in this study. For each procedure, residents were evaluated by means of a form, with one copy being completed by the resident and a separate copy being completed by the attending surgeon. The evaluation form was based on the validated Objective Structured Assessment of Technical Skills for Surgery (OSATS) and used a 5-point Likert-type scale with 7 categories: respect for tissue; time and motion; instrument handling; knowledge of instruments; flow of operation; use of assistants; and knowledge of specific procedure. Data were then stratified by faculty versus resident (self-) assessment; postgraduate year level; and difficulty of procedure. Descriptive statistics (means and SDs) were calculated, and the results were compared using the Wilcoxon signed-rank test and Student t-test. A p value < 0.05 was considered statistically significant. RESULTS Six faculty members, 1 fellow, and 8 residents completed evaluations for 299 procedures, including 32 ventriculoperitoneal (VP) shunt revisions, 23 VP shunt placements, 19 endoscopic third ventriculostomies, and 18 craniotomies for tumor resection. There was no significant difference between faculty and resident self-assessment scores overall or in any of the 7 domains scores for each of the involved residents. On self-assessment, senior residents scored themselves significantly higher (p < 0.02) than junior residents overall and in all domains except for “time and motion.” Faculty members scored senior residents significantly higher than junior residents only for the “knowledge of instruments” domain (p = 0.05). When procedure difficulty was considered, senior residents’ scores from faculty members were significantly higher (p = 0.04) than the scores given to junior residents for expert procedures only. Senior residents’ self-evaluation scores were significantly higher than those of junior residents for both expert (p = 0.03) and novice (p = 0.006) procedures. CONCLUSIONS OSATS is a feasible and reliable assessment tool for the comprehensive evaluation of neurosurgery resident performance in the operating room. The authors plan to use this tool to assess resident operative skill development and to improve direct resident feedback.

show abstract

Utilizing a Novel Tool for the Comprehensive Assessment of Resident Operative Performance

Cited by 42 publications

References 14 publications

Validation of an oncoplastic breast simulator for assessment of technical skills in wide local excision

Validation of an oncoplastic breast simulator for assessment of technical skills in wide local excision

Integrating Postoperative Feedback Into Workflow: Perceived Practices and Barriers

Use of a formal assessment instrument for evaluation of resident operative skills in pediatric neurosurgery

Contact Info

Product

Resources

About