Automatic text classification of actionable radiology reports of tinnitus patients using bidirectional encoder representations from transformer (BERT) and in-domain pre-training (IDPT)

Abstract

Background Given the increasing number of people suffering from tinnitus, the accurate categorization of patients with actionable reports is attractive in assisting clinical decision making. However, this process requires experienced physicians and significant human labor. Natural language processing (NLP) has shown great potential in big data analytics of medical texts; yet, its application to domain-specific analysis of radiology reports is limited. Objective The aim of this study is to propose a novel approach in classifying actionable radiology reports of tinnitus patients using bidirectional encoder representations from transformer BERT-based models and evaluate the benefits of in domain pre-training (IDPT) along with a sequence adaptation strategy. Methods A total of 5864 temporal bone computed tomography(CT) reports are labeled by two experienced radiologists as follows: (1) normal findings without notable lesions; (2) notable lesions but uncorrelated to tinnitus; and (3) at least one lesion considered as potential cause of tinnitus. We then constructed a framework consisting of deep learning (DL) neural networks and self-supervised BERT models. A tinnitus domain-specific corpus is used to pre-train the BERT model to further improve its embedding weights. In addition, we conducted an experiment to evaluate multiple groups of max sequence length settings in BERT to reduce the excessive quantity of calculations. After a comprehensive comparison of all metrics, we determined the most promising approach through the performance comparison of F1-scores and AUC values. Results In the first experiment, the BERT finetune model achieved a more promising result (AUC-0.868, F1-0.760) compared with that of the Word2Vec-based models(AUC-0.767, F1-0.733) on validation data. In the second experiment, the BERT in-domain pre-training model (AUC-0.948, F1-0.841) performed significantly better than the BERT based model(AUC-0.868, F1-0.760). Additionally, in the variants of BERT fine-tuning models, Mengzi achieved the highest AUC of 0.878 (F1-0.764). Finally, we found that the BERT max-sequence-length of 128 tokens achieved an AUC of 0.866 (F1-0.736), which is almost equal to the BERT max-sequence-length of 512 tokens (AUC-0.868,F1-0.760). Conclusion In conclusion, we developed a reliable BERT-based framework for tinnitus diagnosis from Chinese radiology reports, along with a sequence adaptation strategy to reduce computational resources while maintaining accuracy. The findings could provide a reference for NLP development in Chinese radiology reports.

Applications of Artificial Intelligence in Temporal Bone Imaging: Advances and Future Challenges

“…The study of Li et al aimed to propose the use of a novel tool in actionable radiology reports classification in individuals with tinnitus [ 62 ]. This approach uses bidirectional encoder representations derived from BERT-based software.…”

“…The interpretation of 5864 CT scans was initially conducted by two radiologists and then compared to a deep-learning neural network’s performance. In comparison to the Word2vec-based models, the BERT-based model showed a superior result (AUC: 0.868, F1: 0.760) [ 62 ].…”

An interpretable deep learning framework for predicting liver metastases in postoperative colorectal cancer patients using natural language processing and clinical data integration

“…More recently, Causa Andrieu et al 15 developed an NLP‐based radiology report analysis model to identify clinically meaningful CRC metastatic phenotypes and demonstrated a correlation between the phenotypes and overall clinical survival. Our previous study established a domain‐specific transfer learning pipeline to identify patients with clinically meaningful pathogenesis related to tinnitus 16 . However, the integration of multimodal data, such as free text, genomics, and radiomics, and structured data has always been a critical challenge in modeling.…”

“…Our previous study established a domain‐specific transfer learning pipeline to identify patients with clinically meaningful pathogenesis related to tinnitus. 16 However, the integration of multimodal data, such as free text, genomics, and radiomics, and structured data has always been a critical challenge in modeling.…”

Using a classification model for determining the value of liver radiological reports of patients with colorectal cancer

“…Therefore, the rule-based NLP method was used to extract information from imaging reports written by specific templates in Chinese, just such as breast cancer ( 8 , 9 ). The NLP model based on machine learning is used in more researches on imaging reports in Chinese, and good results have been achieved ( 10 , 11 ).…”