Impact of a financial incentive on the completion of educational metrics

Abstract

Background The Accreditation Council for Graduate Medical Education (ACGME) requires all emergency medicine (EM) training programs to evaluate resident performance and also requires core faculty to attend didactic conference. Assuring faculty participation in these activities can be challenging. Previously, our institution did not have a formal tracking program nor financial incentive for participation in these activities. In 2017, we initiated an educational dashboard which tracked and published all full-time university faculty conference attendance and participation in resident evaluations and other educational activities. Objectives We sought to determine if the implementation of a financially-incentivized educational dashboard would lead to an increase in faculty conference attendance and the number of completed resident evaluations. Methods We conducted a pre- and post-intervention observational study at our EM residency training program between July 2017 and July 2019. Participants were 17 full-time EM attendings at one training site. We compared the number of completed online resident evaluations (MedHub) and number of conference days attended (call-in verification) before and after the introduction of our financial incentive in June 2018. The incentive required 100% completion of resident evaluations and at least 25% attendance at eligible didactic conference days. We calculated pre- and post-intervention averages, and comparisons were made using a chi-square test. Results Prior to implementation of the intervention, the 90-day resident evaluation completion rate was 71.8%. This increased to 100% after implementation (p < 0.001). Conference attendance prior to implementation was 43.8%, which remained unchanged at 41.3% after implementation of the financial incentive (p = 0.920). Conclusions Attaching a financial incentive to a tracked educational dashboard increased faculty participation in resident evaluations but did not change conference attendance. This difference likely reflects the minimum thresholds required to obtain the financial incentive.

Educational Relative Value Units as a Measure of Academic Productivity: A Systematic Review

“…In the second of the five papers, Pugh et al reported that providers who achieved both of the pre-set benchmarks for evaluation completion and attendance were to be monetarily incentivized each quarter [ 22 ]. Faculty significantly achieved/exceeded expectations for percent completion of resident evaluations and did so in a shorter time frame.…”

“…Many, it seemed, were published as proof of concept papers, demonstrating that an MBM approach to the education mission was possible, albeit complex. Others demonstrated some improvements in specific aspects of an educational program (e.g., completion of learner evaluations [ 22 ]) associated with some effort to incentivize the desired behaviors. The nature of the papers was such that often there was not a clear methods section, so we had to infer that certain measures were made, even if they were not described.…”

“…It also does not take into consideration quality of teaching. Yes 65: Impact of a financial incentive on the completion of educational metrics, Pugh A, Internatl J Emerg Med, 2020 [ 22 ] Quantitative Department(s) Emergency Med (1) 17 Attaching a financial incentive to a tracked educational dashboard increased faculty participation in resident evaluations but did not change conference attendance. No Not reported Completed evals 71.8% to 100%, conference attendance dropped (had to meet quarterly incentive to complete 100% of resident evaluations within 90 days) No Not reported 2.5% of annual salary (~$4,500) if minimum achieved Not clear in paper, seems to imply incentive was for complete evals AND attendance; only the former was achieved, but paper implies that $ was allocated Not reported Not reported Not reported Not reported Not reported Not reported Not reported Not reported Not reported Not reported Not reported Not reported Not reported Not reported Not reported Conference attendance: achieve at least 25% attendance at eligible didactic conference days (deemed eligible if not scheduled for other conflicting clinical or educational activities (eg clinical shift).…”

“…Pugh [22] Administrative measures improved 90-day resident evaluation completion rate 71.8% ®100%. Mean conference attendance unchanged.…”

Commentary: Case Curve: A Novel Web-Based Platform and Mobile Phone Application to Evaluate Surgical Competence in Graduate Medical Education

“…The article also underscores the importance of addressing faculty concerns, introducing incentives, and incorporating machine learning to enhance the tool's efficiency and overcome current limitations. [4][5][6] The study's strengths lie in the innovative approach, comprehensive methodology, and the potential to improve surgical education through improved feedback with this technology. However, for a larger implementation, this proposal needs to overcome significant challenges including faculty resistance, the need for standardization, and ensuring diversity in faculty participation.…”