Practical Guide to Natural Language Processing for                     Radiology

Mozayan, Ali; Fabbri, Alexander R.; Maneevese, Michelle; Tocino, Irena; Chheang, Sophie

doi:10.1148/rg.2021200113

Cited by 27 publications

(10 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Python is the most frequently used programming language for deep learning worldwide (17,18). Besides Python, Java and C++ are also used occasionally for NLP based research (18). The popularity of a programming language can be found on the TIOBE website (19).…”

Section: Adjudicator Agreement Rate (Aar) = Number Of Cases Where Adj...mentioning

confidence: 99%

See 1 more Smart Citation

Leveraging LLMs like ChatGPT for robust quality checks and medical text agreement rationale, enhancing adjudication quality and alignment in BICR for oncology clinical trials

Sharma,

Farough,

Burkett

et al. 2024

Medical Imaging 2024: Imaging Informatics for Healthcare, Research, and Applications

View full text Add to dashboard Cite

Purpose: Blinded independent central review (BICR) is recommended by the US FDA for registration of oncology trials as image assessment bias is avoided and no chance of unblinding of patient data. Double read with adjudication is the method used to reduce endpoint assessment variability. In cases of disagreement between the readers, a third reader called an adjudicator, reviews the assessment by the two radiologists and decides which assessment is most accurate. Adjudication rate (AR) and adjudicator agreement rate (AAR) are the two indicators used to evaluate reviewer performance and overall trial variability and quality. Sentiment analysis (SA) is based on natural language processing and can tag the data as 'positive', 'negative' or 'neutral' although current technologies can provide a more complex analysis of emotions in the written text. Medical SA can analyze patients' and doctors' opinions, sentiments, attitudes, and emotions in the clinical background. Python, the most frequently used programming language for deep learning worldwide and ChatGPT, an AI-based chatbot can be used for assessing adjudicator comment quality based on sentiment analysis. If successful, this analysis can open another novel implementation for Large Language Models (LLMs) or ChatGPT in clinical research and medical imaging. Methods: This prospective study involved the review of cases for 100 subjects by board-certified radiologists using the Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 criteria. The study employed a double read with adjudication paradigm in a central imaging review setup. The agreement of adjudication was assessed and compared with the overall response, agreed reader, and medical text. The medical text entered by the adjudicator is usually a free text field that typically lacks standardization and control over its content, which may affect its correlation with reviewer selection for agreement. Although uncommon, errors by the adjudicator can occur due to ambiguous text, mis-clicks, or application delay errors. To analyze the adjudicator's comments, sentiment analysis was conducted using a Python plug-in with ChatGPT as a large language model. Based on this analysis, the subjects were categorized as either having "Potential Error" or "No Error". Results: The algorithm supported by ChatGPT was evaluated against a Gold Standard, determined by a board-certified radiologist with over 20 years of experience in the BICR process. A comparison was made to assess accuracy and reproducibility, revealing that only 4 out of 100 subjects had different outcomes. The sensitivity was calculated as 0.857, specificity as 1.0, and accuracy as 0.96. Conclusions:The remarkable Natural Language Processing (NLP) capabilities of ChatGPT are evident in its ability to classify the sentiment as positive, negative, or neutral based on the free-text adjudicator comments provided during the review process. This classification enables a comparison with the actual assessment, adjudicator agreement, and overall patient outcome, highlight...

show abstract

Section: Adjudicator Agreement Rate (Aar) = Number Of Cases Where Adj...mentioning

confidence: 99%

“…Due to its comprehensive standard library that provides simplicity, ease of use and vast amounts of freely available open-source tools, Python remains widely used software for NLP research. Additionally, it has abundant software bundles called packages for NLP applications (18) (20).…”

Section: Adjudicator Agreement Rate (Aar) = Number Of Cases Where Adj...mentioning

confidence: 99%

Leveraging LLMs like ChatGPT for robust quality checks and medical text agreement rationale, enhancing adjudication quality and alignment in BICR for oncology clinical trials

Sharma,

Farough,

Burkett

et al. 2024

Medical Imaging 2024: Imaging Informatics for Healthcare, Research, and Applications

View full text Add to dashboard Cite

show abstract

“…BERT, which builds on the Transformer model, introduces bidirectional training to language modeling. This means that the model can learn the context of a word based on the words that come before and after it in a sentence, resulting in a much deeper understanding of language context and flow than traditional single-direction models [ 39 ]. To further enhance their performance, researchers have adopted transfer learning, a technique where a pre-trained model is fine-tuned on a specific task.…”

Section: Introductionmentioning

confidence: 99%

Natural Language Processing for Breast Imaging: A Systematic Review

Diab

Deng

et al. 2023

Diagnostics

View full text Add to dashboard Cite

Natural Language Processing (NLP) has gained prominence in diagnostic radiology, offering a promising tool for improving breast imaging triage, diagnosis, lesion characterization, and treatment management in breast cancer and other breast diseases. This review provides a comprehensive overview of recent advances in NLP for breast imaging, covering the main techniques and applications in this field. Specifically, we discuss various NLP methods used to extract relevant information from clinical notes, radiology reports, and pathology reports and their potential impact on the accuracy and efficiency of breast imaging. In addition, we reviewed the state-of-the-art in NLP-based decision support systems for breast imaging, highlighting the challenges and opportunities of NLP applications for breast imaging in the future. Overall, this review underscores the potential of NLP in enhancing breast imaging care and offers insights for clinicians and researchers interested in this exciting and rapidly evolving field.

show abstract

“…Radiology reports are constructed with domain-specific terms and patterns, and most of them contain unstructured data [ 6 ]. The typical format of a free-text radiology report consists of four sections: the demographics section describes basic information such as the patient’s name, age, gender, etc., the clinical information section refers to the medical history or current syndrome.…”

Section: Introductionmentioning

confidence: 99%

Automatic text classification of actionable radiology reports of tinnitus patients using bidirectional encoder representations from transformer (BERT) and in-domain pre-training (IDPT)

Lin

Zhao

et al. 2022

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

Background Given the increasing number of people suffering from tinnitus, the accurate categorization of patients with actionable reports is attractive in assisting clinical decision making. However, this process requires experienced physicians and significant human labor. Natural language processing (NLP) has shown great potential in big data analytics of medical texts; yet, its application to domain-specific analysis of radiology reports is limited. Objective The aim of this study is to propose a novel approach in classifying actionable radiology reports of tinnitus patients using bidirectional encoder representations from transformer BERT-based models and evaluate the benefits of in domain pre-training (IDPT) along with a sequence adaptation strategy. Methods A total of 5864 temporal bone computed tomography(CT) reports are labeled by two experienced radiologists as follows: (1) normal findings without notable lesions; (2) notable lesions but uncorrelated to tinnitus; and (3) at least one lesion considered as potential cause of tinnitus. We then constructed a framework consisting of deep learning (DL) neural networks and self-supervised BERT models. A tinnitus domain-specific corpus is used to pre-train the BERT model to further improve its embedding weights. In addition, we conducted an experiment to evaluate multiple groups of max sequence length settings in BERT to reduce the excessive quantity of calculations. After a comprehensive comparison of all metrics, we determined the most promising approach through the performance comparison of F1-scores and AUC values. Results In the first experiment, the BERT finetune model achieved a more promising result (AUC-0.868, F1-0.760) compared with that of the Word2Vec-based models(AUC-0.767, F1-0.733) on validation data. In the second experiment, the BERT in-domain pre-training model (AUC-0.948, F1-0.841) performed significantly better than the BERT based model(AUC-0.868, F1-0.760). Additionally, in the variants of BERT fine-tuning models, Mengzi achieved the highest AUC of 0.878 (F1-0.764). Finally, we found that the BERT max-sequence-length of 128 tokens achieved an AUC of 0.866 (F1-0.736), which is almost equal to the BERT max-sequence-length of 512 tokens (AUC-0.868,F1-0.760). Conclusion In conclusion, we developed a reliable BERT-based framework for tinnitus diagnosis from Chinese radiology reports, along with a sequence adaptation strategy to reduce computational resources while maintaining accuracy. The findings could provide a reference for NLP development in Chinese radiology reports.

show abstract

Practical Guide to Natural Language Processing for Radiology

Cited by 27 publications

References 12 publications

Leveraging LLMs like ChatGPT for robust quality checks and medical text agreement rationale, enhancing adjudication quality and alignment in BICR for oncology clinical trials

Leveraging LLMs like ChatGPT for robust quality checks and medical text agreement rationale, enhancing adjudication quality and alignment in BICR for oncology clinical trials

Natural Language Processing for Breast Imaging: A Systematic Review

Automatic text classification of actionable radiology reports of tinnitus patients using bidirectional encoder representations from transformer (BERT) and in-domain pre-training (IDPT)

Contact Info

Product

Resources

About