Weakly supervised natural language processing for assessing patient-centered outcome following prostate cancer treatment

Abstract

Background The population-based assessment of patient-centered outcomes (PCOs) has been limited by the efficient and accurate collection of these data. Natural language processing (NLP) pipelines can determine whether a clinical note within an electronic medical record contains evidence on these data. We present and demonstrate the accuracy of an NLP pipeline that targets to assess the presence, absence, or risk discussion of two important PCOs following prostate cancer treatment: urinary incontinence (UI) and bowel dysfunction (BD). Methods We propose a weakly supervised NLP approach which annotates electronic medical record clinical notes without requiring manual chart review. A weighted function of neural word embedding was used to create a sentence-level vector representation of relevant expressions extracted from the clinical notes. Sentence vectors were used as input for a multinomial logistic model, with output being either presence, absence or risk discussion of UI/BD. The classifier was trained based on automated sentence annotation depending only on domain-specific dictionaries (weak supervision). Results The model achieved an average F1 score of 0.86 for the sentence-level, three-tier classification task (presence/absence/risk) in both UI and BD. The model also outperformed a pre-existing rule-based model for note-level annotation of UI with significant margin. Conclusions We demonstrate a machine learning method to categorize clinical notes based on important PCOs that trains a classifier on sentence vector representations labeled with a domain-specific dictionary, which eliminates the need for manual engineering of linguistic rules or manual chart review for extracting the PCOs. The weakly supervised NLP pipeline showed promising sensitivity and specificity for identifying important PCOs in unstructured clinical text notes compared to rule-based algorithms. Trial registration This is a chart review study and approved by Institutional Review Board (IRB).

Phenotyping severity of patient‐centered outcomes using clinical notes: A prostate cancer use case

“…In addition, CNN model was less successful at identifying the "severe" category compared to the mild and moderate Both rule-based and machine learning NLP approaches to leverage granular data in EHRs are common and their accuracy has been demonstrated in many recent studies. 5,6,22 In our study, the rulebased approach depends on human expertise outperformed the machine learning approach on UI severity extraction task. As reported in other studies, 9,22,23,[25][26][27] the underlying reason for this may be that the hand-designed rules that precisely capture specific patterns overfit with the data.…”

“…A window size of 8 was chosen since these hyperparameters have been shown to work well in other studies utilizing word vectors. 6,22 The model was trained for 10 epochs using the Gensim python library implementation of word2vec. 23 The trained vectors have the desirable property of assigning semantically similar words close together in Euclidean space.…”

“…As a second cohort filtering, we used a previously described machine learning method to categorize patients into affirmed and negated classification of UI. 6 The present study used only those notes that were categorized as affirmed UI (0.90 F1 score) by this algorithm to reduce computation time. This method yielded a total of 19 213 notes of 3612 patients with predicted affirmed UI (Figure 1).…”

<p>Clinical Documentation to Predict Factors Associated with Urinary Incontinence Following Prostatectomy for Prostate Cancer</p>

“…Regardless of performance, the model will understate the true prevalence of UI if either clinicians or patients do not report all symptoms, or if the manner in which UI is documented is not captured by the design of the pipeline. 13 Second, this study was performed at a single institution, which limits the generalizability and the power of this analysis. Future studies in other healthcare settings and institutions could allow us to assess the reproducibility of NLP-derived findings given potential variability in physician documentation and patient population.…”

“…We assessed UI for each patient using an NLP pipeline that annotates EHR free-text notes as previously reported. 13 This open-source pipeline cleans EHR free-text notes and extracts sentences containing at least one of 61 unique terms indicative of urinary incontinence, a dictionary that was created by a group of urology professionals at our institution (Supplement Table 2). The pipeline does not require manually labeled text for training the NLP model.…”