Automated labelling of radiology reports using natural language processing: Comparison of traditional and newer methods

Abstract

Automated labelling of radiology reports using natural language processing allows for the labelling of ground truth for large datasets of radiological studies that are required for training of computer vision models. This paper explains the necessary data preprocessing steps, reviews the main methods for automated labelling and compares their performance. There are four main methods of automated labelling, namely: (1) rules‐based text‐matching algorithms, (2) conventional machine learning models, (3) neural network models and (4) Bidirectional Encoder Representations from Transformers (BERT) models. Rules‐based labellers perform a brute force search against manually curated keywords and are able to achieve high F1 scores. However, they require proper handling of negative words. Machine learning models require preprocessing that involves tokenization and vectorization of text into numerical vectors. Multilabel classification approaches are required in labelling radiology reports and conventional models can achieve good performance if they have large enough training sets. Deep learning models make use of connected neural networks, often a long short‐term memory network, and are similarly able to achieve good performance if trained on a large data set. BERT is a transformer‐based model that utilizes attention. Pretrained BERT models only require fine‐tuning with small data sets. In particular, domain‐specific BERT models can achieve superior performance compared with the other methods for automated labelling.

Efficiency, accuracy, and health professional's perspectives regarding artificial intelligence in radiology practice: A scoping review

“…In recent years, artificial intelligence (AI) has emerged as a rapidly advancing technology with great potential in the field of medicine. One area where AI is showing particular promise is in clinical radiology [1,2], which is widely applied for many medical conditions like breast cancer detection [3], lung screening [4], brain imaging analysis [5], and more.The recent development of pretrained bidirectional encoder representations from transformers (BERT) models in natural language processing (NLP) further enhanced the performance of automated labeling [6]. These innovations are enabling AI-driven diagnostic competency that could fundamentally transform radiological practices [7][8][9].…”

Knowledge graph-based small sample learning for label of medical imaging reports

“…Arti cial intelligence is becoming increasingly crucial in the eld of radiology diagnosis, providing various possibilities to reduce the workload of radiologists [1]. To develop machine learning or deep learning models for tasks such as image classi cation, object detection, or image segmentation [2][3][4], a substantial labeled dataset is essential. Yet, manually labeling a large image dataset is a time-consuming task for radiologists.…”