Deep learning algorithms to identify documentation of serious illness conversations during intensive care unit admissions

Abstract

Background: Timely documentation of care preferences is an endorsed quality indicator for seriously ill patients admitted to intensive care units. Clinicians document their conversations about these preferences as unstructured free text in clinical notes from electronic health records. Aim: To apply deep learning algorithms for automated identification of serious illness conversations documented in physician notes during intensive care unit admissions. Design: Using a retrospective dataset of physician notes, clinicians annotated all text documenting patient care preferences (goals of care or code status limitations), communication with family, and full code status. Clinician-coded text was used to train algorithms to identify documentation and to validate algorithms. The validated algorithms were deployed to assess the percentage of intensive care unit admissions of patients aged ⩾75 that had care preferences documented within the first 48 h. Setting/participants: Patients admitted to one of five intensive care units. Results: Algorithm performance was calculated by comparing machine-identified documentation to clinician-coded documentation. For detecting care preference documentation at the note level, the algorithm had F1-score of 0.92 (95% confidence interval, 0.89 to 0.95), sensitivity of 93.5% (95% confidence interval, 90.0% to 98.0%), and specificity of 91.0% (95% confidence interval, 86.4% to 95.3%). Applied to 1350 admissions of patients aged ⩾75, we found that 64.7% of patient intensive care unit admissions had care preferences documented within the first 48 h. Conclusion: Deep learning algorithms identified patient care preference documentation with sensitivity and specificity approaching that of clinicians and computed in a tiny fraction of time. Future research should determine the generalizability of these methods in multiple healthcare systems.

Randomized Trial of a Palliative Care Intervention to Improve End-of-Life Care Discussions in Patients With Metastatic Breast Cancer

“…To identify clinician documentation of EoL care discussions, we used natural language processing (NLP) methods to query the EHR with a validated algorithm identifying terms for goals of care and EoL discussions. [29][30][31][32] The NLP software, ClinicalRegex, displays clinical notes that contain phrases about EoL discussions. An independent coder (ie, an oncologist blind to group assignment) reviewed the documentation highlighted by NLP to ensure accurate identification of EoL care discussions.…”

Long-Term Prognosis of Older Adults Who Survive Emergency Mechanical Ventilation

“…To determine the presence of serious illness conversations, we utilized a previously validated neural network described by Chan et al to identify the documentation of such conversations among free-text clinician notes. 22 The network was developed using machine learning algorithms to identify phrases associated with care preference documentation in these notes and demonstrated a sensitivity of 93.5% and specificity of 91.0% when compared to manual review by physicians. Only notes written by physicians were included in the search.…”

“…9,14,21 Specifically, the age cutoff of 75 was chosen to remain consistent with the NQF care measure 9 , which sites the ACOVE guidelines' definition of a "vulnerable elder" as an individual 75 years of age or older 14 , and to remain consistent with the deep learning algorithm, which has only been validated in individuals 75 and older. 22 In addition, individuals who undergo early mechanical ventilation are less likely than other ICU patients to receive serious illness conversations. 23 We chose emergent admissions order to focus on critically ill older adults and to exclude those intubated and admitted for less emergent reasons (i.e., following elective surgery).…”

A Corpus for Detecting High-Context Medical Conditions in Intensive Care Patient Notes Focusing on Frequently Readmitted Patients

“…While this form of high context information may not be found in the structured EHR data, it may be accessible in patient notes, including nursing progress notes and discharge summaries, particularly through the utilization of natural language processing (NLP) technologies. (Chan et al, 2019), (Moon et al, 2019) Given progress in NLP methods, we sought to address the issue of unstructured clinical text by defining and annotating clinical phenotypes in text which may otherwise be prohibitively difficult to discern in the structured data associated with the text entry. For this task, we chose the notes present in the publicly available MIMIC database (Johnson et al, 2016).…”