Deep learning in clinical natural language processing: a methodical review

Abstract

Objective This article methodically reviews the literature on deep learning (DL) for natural language processing (NLP) in the clinical domain, providing quantitative analysis to answer 3 research questions concerning methods, scope, and context of current research. Materials and Methods We searched MEDLINE, EMBASE, Scopus, the Association for Computing Machinery Digital Library, and the Association for Computational Linguistics Anthology for articles using DL-based approaches to NLP problems in electronic health records. After screening 1,737 articles, we collected data on 25 variables across 212 papers. Results DL in clinical NLP publications more than doubled each year, through 2018. Recurrent neural networks (60.8%) and word2vec embeddings (74.1%) were the most popular methods; the information extraction tasks of text classification, named entity recognition, and relation extraction were dominant (89.2%). However, there was a “long tail” of other methods and specific tasks. Most contributions were methodological variants or applications, but 20.8% were new methods of some kind. The earliest adopters were in the NLP community, but the medical informatics community was the most prolific. Discussion Our analysis shows growing acceptance of deep learning as a baseline for NLP research, and of DL-based NLP in the medical community. A number of common associations were substantiated (eg, the preference of recurrent neural networks for sequence-labeling named entity recognition), while others were surprisingly nuanced (eg, the scarcity of French language clinical NLP with deep learning). Conclusion Deep learning has not yet fully penetrated clinical NLP and is growing rapidly. This review highlighted both the popular and unique trends in this active field.

Machine learning and natural language processing methods to identify ischemic stroke, acuity and location from radiology reports

“…This investigation is part of a wider literature that employs deep learning in clinical NLP [31]. In this study, we employ a specific RNN structure that had been previously and successfully used in combination with GloVe embeddings [16].…”

Modern Clinical Text Mining: A Guide and Review

“…Most of the criticisms of EHRs in recent years have focused on their role in physician burnout (146,147). The technology perspective on EHR data, fueled by methodological advances like deep learning (23) and the near-continuous development of high-performing predictive and diagnostic algorithms (6,Table 3), has primarily been one of excitement.…”

“…Until the last few years, embeddings consisted of one vector per entity; that is, one vector per word, phrase, or document. However, novel neural network architectures (23) have permitted the creation of embeddings that vary depending on the context; this has expanded the representational power of embedding methods and led to the creation of massive pretrained language models like BERT (Bidirectional Encoder Representations from Transformers) (30). These models are generally too resource-intensive to be trained from scratch.…”

“…Several studies have deployed convolutional neural networks (CNNs) with high success on a variety of clinical text classification tasks: assigning diagnosis codes (104,105), classifying radiology reports (19,106), subtyping diseases (91), and determining the presence or absence of comorbidities (107). Alternative neural network architectures, such as LSTMs and attention networks, are commonly used in text classification tasks in the general NLP domain, although as of this writing, CNNs have been the dominant architecture in clinical text classification (23,108). One recent paper exemplifies the end-to-end deep learning approach to clinical text classification, tying rule-based features together with word and UMLS-based concept embeddings in a single CNN-based classifier (107).…”

A Gated Dilated Convolution with Attention Model for Clinical Cloze-Style Reading Comprehension

“…Some research communities began to launch competitions on machine reading comprehension in the clinical field, such as MEDIQA 2019 [4], BIOASQ [5], etc. These tasks have attracted some researchers to carry out various researches, and have played dramatic roles in promoting researches in the clinical medical field [6]. And some related data sets have been proposed, such as CliCR [7], PubMedQA [8], Chimed [2] and emrQA [3] etc.…”