A novel deep learning approach to extract Chinese clinical entities for lung cancer screening and staging

Abstract

Background Computed tomography (CT) reports record a large volume of valuable information about patients’ conditions and the interpretations of radiology images from radiologists, which can be used for clinical decision-making and further academic study. However, the free-text nature of clinical reports is a critical barrier to use this data more effectively. In this study, we investigate a novel deep learning method to extract entities from Chinese CT reports for lung cancer screening and TNM staging. Methods The proposed approach presents a new named entity recognition algorithm, namely the BERT-based-BiLSTM-Transformer network (BERT-BTN) with pre-training, to extract clinical entities for lung cancer screening and staging. Specifically, instead of traditional word embedding methods, BERT is applied to learn the deep semantic representations of characters. Following the long short-term memory layer, a Transformer layer is added to capture the global dependencies between characters. Besides, pre-training technique is employed to alleviate the problem of insufficient labeled data. Results We verify the effectiveness of the proposed approach on a clinical dataset containing 359 CT reports collected from the Department of Thoracic Surgery II of Peking University Cancer Hospital. The experimental results show that the proposed approach achieves an 85.96% macro-F1 score under exact match scheme, which improves the performance by 1.38%, 1.84%, 3.81%,4.29%,5.12%,5.29% and 8.84% compared to BERT-BTN, BERT-LSTM, BERT-fine-tune, BERT-Transformer, FastText-BTN, FastText-BiLSTM and FastText-Transformer, respectively. Conclusions In this study, we developed a novel deep learning method, i.e., BERT-BTN with pre-training, to extract the clinical entities from Chinese CT reports. The experimental results indicate that the proposed approach can efficiently recognize various clinical entities about lung cancer screening and staging, which shows the potential for further clinical decision-making and academic research.

Automatic text classification of actionable radiology reports of tinnitus patients using bidirectional encoder representations from transformer (BERT) and in-domain pre-training (IDPT)

“…Pre-training is an important technique in NLP field, this approach has recently attracted increasing attention, especially in healthcare related fields. For instance, Zhang et al [41] designed and evaluated the feasibility of using pre-training models to extract key information from Chinese radiology reports fort lung cancer staging, the model achieved an F1 of 85.96%,while our study achieved an F1 of 84.10%. More recently, Nakamura et al [39] applied BERT without IDPT to classify actionable Japanese radiology reports, and attempted to predict a positive/negative "actionable tag", the results seem promising with highest AUC of 0.95.…”

Automatic symptoms identification from a massive volume of unstructured medical consultations using deep neural and BERT models

“…When the data is textual, the extraction of features is a bit different where the aim is to create word or text embeddings. Generally in medical NLP and at the feature extraction level, the BERT (as a state-of-the-art model) was used in several studies as in [14] , [15] , [16] , [17] , whereas, the Word2Vec model was used in [18] , [19] , [20] . In contrast, and at the algorithmic level, various deep learning models were widely used in medical NLP.…”

“…In contrast, and at the algorithmic level, various deep learning models were widely used in medical NLP. Such as: The BiLSTM/LSTM in [14] , [16] , [20] , [21] , [22] , [23] , [24] , the convolutional neural network (CNN) in [19] , [25] , the capsule network [26] , the transformers [27] , the ResNet-34 network [28] , and the generative adversarial network (GAN) [29] . Medical symptom extraction is a well-known problem in health or medical-related NLP tasks.…”

Automatic Symptoms Identification from Massive Volume of Unstructured Medical Consultations Using Deep Neural and BERT Models

“…When the data is textual, the extraction of features is a bit different where the aim is to create word or text embeddings. Generally in medical NLP and at the feature extraction level, the BERT (as a state-of-the-art model) was used in several studies as in [14,15,16,17], whereas, the Word2Vec model was used in [18,19,20]. In contrast, and at the algorithmic level, various deep learning models were widely used in medical NLP.…”

“…In contrast, and at the algorithmic level, various deep learning models were widely used in medical NLP. Such as: The BiLSTM/LSTM in [14,16,20,21,22,23,24], the convolutional neural network (CNN) in [19,25], the capsule network [26], the transformers [27], the ResNet-34 network [28], and the generative adversarial network (GAN) [29]. Medical symptom extraction is a wellknown problem in health or medical-related NLP tasks.…”