Evaluating Surgical Residents Quickly and Easily Against the Milestones Using Electronic Formative Feedback

Hartranft, Thomas H.; Yandle, Kurt; Graham, Tim; Holden, Charles; Chambers, Lowell W.

doi:10.1016/j.jsurg.2016.09.006

Cited by 10 publications

(6 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…34,35 Other studies have focused on providing tools that can increase the ease with which resident evaluations can be completed, whether using app-based systems or QR codes; these studies have primarily focused on increasing the number of evaluations completed rather than on the feedback itself. [36][37][38][39] While increasing the quantity of feedback may be important, unintended consequences, such as degrading the process into one of "form filling" and "checking boxes," may occur. 40 Most importantly, many of the studies on feedback interventions and tools were conducted outside the ED environment and were limited by their retrospective or qualitative design, with few prospective case-control studies, further highlighting the need for an ED-specific tool.…”

Section: Discussionmentioning

confidence: 99%

“…These studies, however, required dedicated faculty coaching and time commitments for the observations, which may be more challenging to secure in the ED setting [ 33 , 34 ]. Other studies have focused on providing tools that can increase the ease with which resident evaluations can be completed, whether using app-based systems or QR codes; these studies have primarily focused on increasing the number of evaluations completed rather than on the feedback itself [ 35 – 38 ]. While increasing the quantity of feedback may be important, unintended consequences, such as degrading the process into one of “form filling” and “checking boxes,” may occur [ 39 ].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

The integrative feedback tool: assessing a novel feedback tool among emergency medicine residents

Gore¹,

Schiebout²,

Peksa³

et al. 2023

Clin Exp Emerg Med

View full text Add to dashboard Cite

Feedback is critical to the growth of learners. However, feedback quality can be variable in practice. Most feedback tools are generic, with few targeting emergency medicine (EM). We created a feedback tool designed for EM residents, and this study aimed to evaluate the effectiveness of this tool. Methods:This was a single-center, prospective cohort study comparing feedback quality before and after introducing a novel feedback tool. Residents and faculty completed a survey after each shift assessing feedback quality, feedback time, and the number of feedback episodes. Feedback quality was assessed using a composite score from 7 questions, which were each scored 1-5 points (minimum total score, 7 points; maximum, 35 points). Pre-and post-intervention data were analyzed using a mixed-effects model with the treatment of study participants as having correlated random effects. Results:Residents completed 182 surveys and faculty members completed 158 surveys. The use of the tool was associated with improved consistency in the summative score of effective feedback attributes as assessed by residents (P = 0.04) but not by faculty (P = 0.259). However, most of the individual scores for attributes of good feedback did not reach statistical significance. With the tool, residents perceived that faculty spent more time providing feedback (P = 0.04) and that the delivery of feedback was more ongoing throughout the shift (P = 0.02). Faculty felt that the tool allowed for more ongoing feedback (P = 0.002), with no perceived increase in the time spent delivering feedback (P = 0.833). Conclusion:The use of a dedicated tool may help educators provide more meaningful and frequent feedback without impacting the perceived required time needed to provide feedback. Capsule SummaryWhat is already known: Feedback is not always focused on or formally taught as part of graduate medical education, and, as a result, many clinical faculty may not have significant training in the matter. Furthermore, many faculty may not have the time to prioritize keeping up to date with evolving literature in medical education given their other clinical and administrative commitments. This can lead to significant variability in how feedback is delivered and result in learner dissatisfaction with the quality of feedback provided as well as missed opportunities for growth and development. What is new in the current study:To address this need, we developed a novel feedback tool to guide feedback delivery and allow opportunities for integration into the shift. Using a structured tool, residents could identify their specific learning objectives from a full list modeled after the EM Milestones. Informed by Kolb's theory of experiential learning, the residents then receive real-time feedback on specific instances after a patient encounter, alter their practice, and see if any changes they made have been effective. Having the learner choose the specific skills in an organized system, with clearly defined and achievable goals to work on during their shift, may prevent...

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

The integrative feedback tool: assessing a novel feedback tool among emergency medicine residents

Gore¹,

Schiebout²,

Peksa³

et al. 2023

Clin Exp Emerg Med

View full text Add to dashboard Cite

show abstract

“…Web-based 27,138,145,[163][164][165]175 Designed to take a minute to complete in order to facilitate same-day, timely responses in brief narrative comments, these systems were felt to be easy to institute and feasible approach to assessing students, particularly regarding professionalism behavior. These online survey platforms can increase the amount and timeliness of feedback.…”

Section: Incept Mini-pat Tab Em-hs Cat E-toolsmentioning

confidence: 99%

“… 18 Studies have shown that these web-based tools can be beneficial for improving faculty engagement in and frequency of their feedback. 11 , 163 , 164 The timely nature of this feedback also leads to increased satisfaction from learners. 164 However, these platforms can be limited by faculty comfort with, and knowledge of, technology.…”

Section: Giving Feedbackmentioning

confidence: 99%

Feedback in Medical Education: An Evidence-based Guide to Best Practices from the Council of Residency Directors in Emergency Medicine

Natesan¹,

Jordan²,

Sheng³

et al. 2023

WestJEM

View full text Add to dashboard Cite

BACKGROUNDFeedback is an important tool within medical education for the improvement of clinical skills and professional development. 1 However, the emergency department (ED) presents a uniquely complex environment for feedback due to the rapid pace and workflow for patient care, relative lack of privacy, and need for constant task-switching. 1 Incorporating feedback into this environment can negatively impact an emergency medicine (EM) resident's training, with consistent reports of dissatisfaction regarding the quality of feedback received from faculty.

show abstract

“…1,5 There is some evidence that electronic ESA programs improve completion rates above paper forms. [6][7][8] Still, evidence of uptake and utility of computerized ESA programs remains scant. In this study, we examine the implementation of a web-based tool with text message prompts to assist mobile ESA (mESA) in an emergency medicine (EM) residency program.…”

Section: Introductionmentioning

confidence: 99%

Implementation of a Web-Based Tool With Text Message Prompts to Improve End-of-Shift Assessments for Emergency Medicine Residents

Tobias

Sobehart

Doshi

et al. 2020

Journal of Graduate Medical Education

View full text Add to dashboard Cite

Background End-of-shift assessments (ESA) can provide representative data on medical trainee performance but do not occur routinely and are not documented systematically. Objective To evaluate the implementation of a web-based tool with text message prompts to assist mobile ESA (mESA) in an emergency medicine (EM) residency program. Methods mESA used timed text messages to prompt faculty/trainees to expect in-person qualitative ESA in a milestone content area and for the faculty to record descriptive performance data through a web-based platform. We assessed implementation between January 2018 and November 2019 using the RE-AIM framework (reach, effectiveness, adoption, implementation, and maintenance). Results Reach: 96 faculty and 79 trainees participated in the mESA program. Effectiveness: From surveys, approximately 72% of faculty and 58% of trainees reported increases in providing and receiving ESA feedback after program implementation. From ESA submissions, trainees reported receiving in-person feedback on 90% of shifts. Residency leadership confirmed perceived utility of the mESA program. Adoption: mESA prompts were sent on 7792 unique shifts across 4 EDs, all days of week, and different times of day. Faculty electronically submitted ESA feedback on 45% of shifts. Implementation quality: No technological errors occurred. Maintenance: Completion of in-person ESA feedback and electronic submission of feedback by faculty was stable over time. Conclusions We found mixed evidence in support of using a web-based tool with text message prompts for mESA for EM trainees.

show abstract

Evaluating Surgical Residents Quickly and Easily Against the Milestones Using Electronic Formative Feedback

Cited by 10 publications

References 2 publications

The integrative feedback tool: assessing a novel feedback tool among emergency medicine residents

The integrative feedback tool: assessing a novel feedback tool among emergency medicine residents

Feedback in Medical Education: An Evidence-based Guide to Best Practices from the Council of Residency Directors in Emergency Medicine

Implementation of a Web-Based Tool With Text Message Prompts to Improve End-of-Shift Assessments for Emergency Medicine Residents

Contact Info

Product

Resources

About