Eliciting Insights From Chat Logs of the 25X5 Symposium to Reduce Documentation Burden: Novel Application of Topic Modeling (Preprint)

Abstract

BACKGROUND Addressing clinician documentation burden through “targeted solutions” is a growing priority for many organizations including government, academia, and industry. Between January and February 2021, the 25 by 5: Symposium to Reduce Documentation Burden on U.S. Clinicians by 75% (25x5 Symposium) convened across six weekly, two-hour sessions among experts and stakeholders to generate actionable goals for reducing clinician documentation over the next five years. Throughout the virtual symposium, we passively collected—with their knowledge that the content would be deidentified and made publicly available—attendee contributions to a chat functionality. This presented a novel opportunity to synthesize and understand participants’ perceptions and interests from chat messages. We performed a content analysis of 25x5 Symposium chat logs to identify themes on reducing clinician documentation burden. OBJECTIVE The objective of this study was to explore unstructured chat log content from the virtual 25x5 Symposium to elicit latent insights on clinician documentation burden among clinicians, healthcare leaders, and other stakeholders using topic modeling. METHODS Across the six sessions, we captured 1,787 messages among 167 unique chat participants cumulatively; fourteen were private messages not included in the analysis. We implemented a latent Dirichlet allocation (LDA) topic model on the aggregated dataset to identify clinician documentation burden topics mentioned in the chat logs. Coherence scores and manual examination informed optimal model selection. Five domain experts independently and qualitatively assigned descriptive labels to model-identified topics and classified them into higher-level categories, which were finalized through a panel consensus. RESULTS We uncovered ten topics using the LDA model: (1) determining data and documentation needs (23.8%), (2) collectively reassessing documentation requirements in electronic health records (EHR) (14.2%), (3) focusing documentation on patient narrative (9.1%), (4) documentation that adds value (8.3%), (5) regulatory impact on clinician burden (8.0%), (6) improved EHR user interface and design (7.2%), (7) addressing poor usability (6.9%), (8) sharing 25x5 Symposium resources (6.9%), (9) capturing data related to clinician practice (6.4%), and (10) role of quality measures and technology on burnout (6.2%). Among these 10 topics, five high-level categories emerged: consensus building (46.3%), burden sources (20.6%), EHR design (14.1%), patient-centered care (9.1%), and symposium comments (6.9%). CONCLUSIONS We conducted a topic modeling analysis on 25x5 Symposium multi-participant chat logs to explore the feasibility of this novel application and to elicit additional insights on clinician documentation burden among attendees. Based on the results of our LDA analysis, consensus building, burden sources, EHR design and patient-centered care may be important themes to consider when addressing clinician documentation burden. Our findings demonstrate the value of topic modeling in discovering topics associated with clinician documentation burden using unstructured textual content. Topic modeling may be a suitable approach to examine latent themes presented in virtual symposium chat logs.