Clinical named entity recognition and relation extraction using natural language processing of medical free text: A systematic review

Navarro, David Fraile; Ijaz, Kiran; Rezazadegan, Dana; Rahimi-Ardabili, Hania; Dras, Mark; Coiera, Enrico; Berkovsky, Shlomo

doi:10.1016/j.ijmedinf.2023.105122

Cited by 10 publications

(1 citation statement)

References 95 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One registered review describes named entity recognition and relation extraction in clinical documents using NLP. However, this review is neither focused on radiology reports nor LLM, and the search process was last updated on 07/07/2021, potentially missing many of recently published articles regarding the application of LLMs (18). Therefore, as LLMs have only recently gained momentum, a research gap exists and there is no overview of LLM-based approaches to IE from radiology reports available.…”

Section: Introductionmentioning

confidence: 99%

Large language model-based information extraction from free-text radiology reports: a scoping review protocol

Reichenpfader

Mueller

Denecke

2023

Preprint

View full text Add to dashboard Cite

Introduction Radiological imaging is one of the most frequently performed diagnostic tests worldwide. The free text contained in radiology reports is currently only rarely utilized for secondary use, including research and predictive analysis. However, this data might be made available by means of information extraction (IE), based on natural language processing (NLP). Recently, a new approach to NLP, large language models (LLMs), has gained momentum and continues to improve performance. The objective of this scoping review is to show the state of research regarding IE from free-text radiology reports based on LLMs, to investigate applied methods, and to guide future research by showing open challenges and limitations of current approaches. To our knowledge, no systematic nor scoping review of IE of radiology reports, based on LLMs, has been conducted yet. Existing publications are outdated and do not comprise LLM-based models. Methods and analysis This protocol is designed based on the JBI manual for evidence synthesis, chapter 11.2: "Development of a scoping review protocol". Inclusion criteria and a search strategy comprising four databases (PubMed, IEEE Xplore, Web of Science Core Collection, ACM Digital Library) are defined. Furthermore, we describe the screening process, data charting, analysis and presentation of extracted data. Ethics and dissemination This protocol describes the methodology of a scoping literature review and does not comprise research on or with humans, animals or their data. Therefore, no ethical approval is required. After the publication of this protocol and the conduct of the review, its results are going to be published in an open access journal dedicated to biomedical informatics/ digital health.

show abstract

Section: Introductionmentioning

confidence: 99%

Large language model-based information extraction from free-text radiology reports: a scoping review protocol

Reichenpfader

Mueller

Denecke

2023

Preprint

View full text Add to dashboard Cite

show abstract

A case study in applying artificial intelligence-based named entity recognition to develop an automated ophthalmic disease registry

Macri

Teoh

Bacchi

et al. 2023

Graefes Arch Clin Exp Ophthalmol

View full text Add to dashboard Cite

Purpose Advances in artificial intelligence (AI)-based named entity extraction (NER) have improved the ability to extract diagnostic entities from unstructured, narrative, free-text data in electronic health records. However, there is a lack of ready-to-use tools and workflows to encourage the use among clinicians who often lack experience and training in AI. We sought to demonstrate a case study for developing an automated registry of ophthalmic diseases accompanied by a ready-to-use low-code tool for clinicians. Methods We extracted deidentified electronic clinical records from a single centre’s adult outpatient ophthalmology clinic from November 2019 to May 2022. We used a low-code annotation software tool (Prodigy) to annotate diagnoses and train a bespoke spaCy NER model to extract diagnoses and create an ophthalmic disease registry. Results A total of 123,194 diagnostic entities were extracted from 33,455 clinical records. After decapitalisation and removal of non-alphanumeric characters, there were 5070 distinct extracted diagnostic entities. The NER model achieved a precision of 0.8157, recall of 0.8099, and F score of 0.8128. Conclusion We presented a case study using low-code artificial intelligence-based NLP tools to produce an automated ophthalmic disease registry. The workflow created a NER model with a moderate overall ability to extract diagnoses from free-text electronic clinical records. We have produced a ready-to-use tool for clinicians to implement this low-code workflow in their institutions and encourage the uptake of artificial intelligence methods for case finding in electronic health records.

show abstract

A Large Language Model to Detect Negated Expressions in Radiology Reports

Su,

Babore,

Kahn

2024

j Imaging. Inform. med.

View full text Add to dashboard Cite

Natural language processing (NLP) is crucial to extract information accurately from unstructured text to provide insights for clinical decision-making, quality improvement, and medical research. This study compared the performance of a rule-based NLP system and a medical-domain transformer-based model to detect negated concepts in radiology reports. Using a corpus of 984 de-identified radiology reports from a large U.S.-based academic health system (1000 consecutive reports, excluding 16 duplicates), the investigators compared the rule-based medspaCy system and the Clinical Assertion and Negation Classification Bidirectional Encoder Representations from Transformers (CAN-BERT) system to detect negated expressions of terms from RadLex, the Unified Medical Language System Metathesaurus, and the Radiology Gamuts Ontology. Power analysis determined a sample size of 382 terms to achieve α = 0.05 and β = 0.8 for McNemar’s test; based on an estimate of 15% negated terms, 2800 randomly selected terms were annotated manually as negated or not negated. Precision, recall, and F1 of the two models were compared using McNemar’s test. Of the 2800 terms, 387 (13.8%) were negated. For negation detection, medspaCy attained a recall of 0.795, precision of 0.356, and F1 of 0.492. CAN-BERT achieved a recall of 0.785, precision of 0.768, and F1 of 0.777. Although recall was not significantly different, CAN-BERT had significantly better precision (χ2 = 304.64; p < 0.001). The transformer-based CAN-BERT model detected negated terms in radiology reports with high precision and recall; its precision significantly exceeded that of the rule-based medspaCy system. Use of this system will improve data extraction from textual reports to support information retrieval, AI model training, and discovery of causal relationships.

show abstract

Clinical named entity recognition and relation extraction using natural language processing of medical free text: A systematic review

Cited by 10 publications

References 95 publications

Large language model-based information extraction from free-text radiology reports: a scoping review protocol

Large language model-based information extraction from free-text radiology reports: a scoping review protocol

A case study in applying artificial intelligence-based named entity recognition to develop an automated ophthalmic disease registry

A Large Language Model to Detect Negated Expressions in Radiology Reports

Contact Info

Product

Resources

About