A study of deep learning approaches for medication and adverse drug event extraction from clinical text

Abstract

Objective This article presents our approaches to extraction of medications and associated adverse drug events (ADEs) from clinical documents, which is the second track of the 2018 National NLP Clinical Challenges (n2c2) shared task. Materials and Methods The clinical corpus used in this study was from the MIMIC-III database and the organizers annotated 303 documents for training and 202 for testing. Our system consists of 2 components: a named entity recognition (NER) and a relation classification (RC) component. For each component, we implemented deep learning-based approaches (eg, BI-LSTM-CRF) and compared them with traditional machine learning approaches, namely, conditional random fields for NER and support vector machines for RC, respectively. In addition, we developed a deep learning-based joint model that recognizes ADEs and their relations to medications in 1 step using a sequence labeling approach. To further improve the performance, we also investigated different ensemble approaches to generating optimal performance by combining outputs from multiple approaches. Results Our best-performing systems achieved F1 scores of 93.45% for NER, 96.30% for RC, and 89.05% for end-to-end evaluation, which ranked #2, #1, and #1 among all participants, respectively. Additional evaluations show that the deep learning-based approaches did outperform traditional machine learning algorithms in both NER and RC. The joint model that simultaneously recognizes ADEs and their relations to medications also achieved the best performance on RC, indicating its promise for relation extraction. Conclusion In this study, we developed deep learning approaches for extracting medications and their attributes such as ADEs, and demonstrated its superior performance compared with traditional machine learning algorithms, indicating its uses in broader NER and RC tasks in the medical domain.

Medical Information Extraction in the Age of Deep Learning

“…Except for route and ADE, all top scores for NER were achieved on the n2c2 corpus. For drug names, the current SOTA exceeds 95% F 1 score established by Wei et al [62]. As to the subtypes, their system also compares favorably to alternative architectures by a large F 1 margin ranging from 8.6 percentage points (for duration) down to 1.0 (for drug name).…”

“…The n2c2 corpus is the reference dataset for top performance. The group who achieved top F 1 scores for the medication NER problem also performed best for the medication-attribute REX task [62], with extraordinary figures for Frequency, Route, and Dosage relations (in the upper 98% F 1 range), a superior one for the Duration relation (93% F 1 ), and good ones on the (hard to deal with) Adverse and Reason relations (85% F 1 ). Still, the distances to the second-best system for the same corpus (n2c2) are not so pronounced in most cases, ranging by 1 percentage point (for Frequency, Route, Dosage, and Duration), yet increased up to 3 (for Adverse) and 7 (for Reason) percentage points.…”

“…Overall, frequency, route, and dosage recognition reach outstanding F 1 scores in the range of 95 up to 97%, while for duration information top F 1 scores drop remarkably by at least 10 to 20 percentage points. Still, the recognition of ADEs seems to be the hardest task, with the best system by Wunnava et al [67] peaking at around 64% F 1 on MADE 1.0 data (here the top performing system by Wei et al [62] plummets down to 53% F 1 ). Interestingly, ADEs are verbally the least constrained type of natural language utterance compared with all the other entity types considered here.…”

“…Wei et al [62] Gligic et al [63] Zeng et al [64] Unanue et al [65] Li et al [66] Wunnava et al [67] Dandala et al [70] Tao et al [71] Chapman et al [72] Unanue et al [65] NE type Word2vec embeddings self-trained on MIMIC-III [58] Word2vec embeddings self-trained on i2b2 2009 [57] word & character embeddings pre-trained on Wikipedia GloVe [53] pre-trained on CommonCrawl + GloVe self-trained on MIMIC-III [58] pre-trained [16] pre-trained on Wikipedia, EHR notes, and PubMed [68,69] pretrained Notes and consists of 1,092 de-identified EHR notes from 21 cancer patients. Each note was annotated with medication information (drug name, dosage, route, frequency, duration), ADEs, indication (symptom as reason for drug administration), other signs and symptoms, severity (of disease/symptom), and relations among those entities, resulting in 79,000 mention annotations.…”

“…Wei et al [62] Tao et al [71] Gligic et al [63] Li et al [66] Dandala et al [70] Chapman et al [72] Wunnava et al [67] Gligic et al [63] Wei et al [62] Tao et al [71] Wunnava et al [67] Chapman et al [72] Li et al [66] Dandala et al [70] Wei et al [62] Gligic et al [63] Tao et al [71] Wunnava et al [67] Li et al [66] Chapman et al [72] Dandala et al [70] Wei et al [62] Li et al [66] Wunnava et al [67] Dandala et al [70] Chapman et al [72] Gligic et al [63] Tao et al [71] Wunnava et al [67] Yang et al [73] Dandala et al [70] Li et al [66] Wei et al [62] Chapman et al [72] Word2vec embeddings self-trained on MIMIC-III [58] (as feature for CRF: GloVe [53] embeddings self-trained on MIMIC-III [58])…”