A Qualitative Description of Clinician Free-Text Rationales Entered within Accountable Justification Interventions

Abstract

Background Requiring accountable justifications—visible, clinician-recorded explanations for not following a clinical decision support (CDS) alert—has been used to steer clinicians away from potentially guideline-discordant decisions. Understanding themes from justifications across clinical content areas may reveal how clinicians rationalize decisions and could help inform CDS alerts. Methods We conducted a qualitative evaluation of the free-text justifications entered by primary care physicians from three pilot interventions designed to reduce opioid prescribing and, in older adults, high-risk polypharmacy and overtesting. Clinicians encountered alerts when triggering conditions were met within the chart. Clinicians were asked to change their course of action or enter a justification for the action that would be displayed in the chart. We extracted all justifications and grouped justifications with common themes. Two authors independently coded each justification and resolved differences via discussion. Three physicians used a modified Delphi technique to rate the clinical appropriateness of the justifications. Results There were 560 justifications from 50 unique clinicians. We grouped these into three main themes used to justify an action: (1) report of a particular diagnosis or symptom (e.g., for “anxiety” or “acute pain”); (2) provision of further contextual details about the clinical case (e.g., tried and failed alternatives, short-term supply, or chronic medication); and (3) noting communication between clinician and patient (e.g., “risks and benefits discussed”). Most accountable justifications (65%) were of uncertain clinical appropriateness. Conclusion Most justifications clinicians entered across three separate clinical content areas fit within a small number of themes, and these common rationales may aid in the design of effective accountable justification interventions. Justifications varied in terms of level of clinical detail. On their own, most justifications did not clearly represent appropriate clinical decision making.

Developing and Testing a Framework for Coding General Practitioners' Free-Text Diagnoses in Electronic Medical Records - A Reliability Study for Generating Training Data in Natural Language Processing

“…This is consistent with ndings from other studies that have analysed the content of LoFT, showing that non-speci c or insu cient information is common in medical documentation (33)(34)(35)(36) but substantially reduced the yield of LoFT for obtaining coded diagnostic data in our study. Speci cally, ambiguous acronyms or abbreviations (37)(38)(39), unstructured information (39)(40)(41), as well as physicians' and institutional stylistic preferences contribute to non-diagnostic information in free-text diagnoses (42). Raters in our study were notably challenged by non-diagnostic information in LoFT, which manifested itself in an IRA of only 93%, whereas all other codes had IRA ≥98%.…”