Clinician collaboration to improve clinical decision support: the Clickbusters initiative

Abstract

Objective We describe the Clickbusters initiative implemented at Vanderbilt University Medical Center (VUMC), which was designed to improve safety and quality and reduce burnout through the optimization of clinical decision support (CDS) alerts. Materials and Methods We developed a 10-step Clickbusting process and implemented a program that included a curriculum, CDS alert inventory, oversight process, and gamification. We carried out two 3-month rounds of the Clickbusters program at VUMC. We completed descriptive analyses of the changes made to alerts during the process, and of alert firing rates before and after the program. Results Prior to Clickbusters, VUMC had 419 CDS alerts in production, with 488 425 firings (42 982 interruptive) each week. After 2 rounds, the Clickbusters program resulted in detailed, comprehensive reviews of 84 CDS alerts and reduced the number of weekly alert firings by more than 70 000 (15.43%). In addition to the direct improvements in CDS, the initiative also increased user engagement and involvement in CDS. Conclusions At VUMC, the Clickbusters program was successful in optimizing CDS alerts by reducing alert firings and resulting clicks. The program also involved more users in the process of evaluating and improving CDS and helped build a culture of continuous evaluation and improvement of clinical content in the electronic health record.

Documentation dynamics: Note composition, burden, and physician efficiency

“…Yet little is known about these systems, the degree to which they do in fact streamline the process, the usability pain points, or how they can be best designed to maximize efficiency. Future research should leverage randomized trials within organizations as well as natural experiments and evaluations of EHR burden reduction programs to more precisely identify the proximate causes of the note bloat that leads to physician EHR burden and ultimately burnout 47,48 …”

“…Future research should leverage randomized trials within organizations as well as natural experiments and evaluations of EHR burden reduction programs to more precisely identify the proximate causes of the note bloat that leads to physician EHR burden and ultimately burnout. 47,48…”

Assessing the Value of ChatGPT for Clinical Decision Support Optimization

“…[36] Suggested steps include: 1) Human-generated suggestions: recruit CDS experts to generate suggestions for a number of CDS alerts (i.e., the Clickbuster process). [19] 2) Supervised fine-tuning (SFT): Use the selected alerts and human-generated suggestion dataset to fine-tune the pre-trained language model to learn a supervised strategy (SFT model) to generate suggestions for selected alerts. 3) Mimic human preferences: recruit CDS experts to rank the output of the baseline model (i.e., the generated suggestions).…”

“…The copyright holder for this preprint this version posted February 23, 2023. ; https://doi.org/10.1101/2023.02.21.23286254 doi: medRxiv preprint (4 physicians and 1 pharmacist) during the Clickbusters program and grouped them by alert. [19] Within each alert, we randomized the order of the AI-and human-generated suggestions. For the humangenerated suggestions, we reformatted them if they included specific alert identifiers, since humans often included these identifiers but ChatGPT never did (because it did not have access to record IDs).…”

“…The first approach uses human review to optimize alert content, timing, and target audience, [15][16][17] which can reduce 9% to 35% of alerts with no untoward consequences. [18] For example, Vanderbilt University Medical Center (VUMC) conducted the Clickbusters program, [19] where 24 physicians and informaticians reviewed alerts at VUMC using a structured process. They eliminated 70,000 unnecessary weekly alert firings, representing a 15% decrease.…”