Natural language processing to identify ureteric stones in radiology reports

Abstract

Introduction Natural language processing (NLP) is an emerging tool which has the ability to automate data extraction from large volumes of unstructured text. One of the main described uses of NLP in radiology is cohort building for epidemiological studies. This study aims to assess the accuracy of NLP in identifying a group of patients positive for ureteric stones on Computed Tomography – Kidneys, Ureter, Bladder (CT KUB) reports. Methods Retrospective review of all CT KUB reports in a single calendar year. A locally available NLP tool was used to automatically classify the reports based on positivity for ureteric stones. This was validated by manual review and refined to maximize the accuracy of stone detection. Results A total of 1874 CT KUB reports were identified. Manual classification of ureteric stone positivity was 36% compared with 27% using NLP. The accuracy of NLP was 85% with a sensitivity of 66% and specificity of 95%. Incorrect classification was due to misspellings, variable syntax, terminology, pluralization and the inability to exclude clinical request details from the search algorithm. Conclusions Our NLP tool demonstrated high specificity but low sensitivity at identifying CT KUB reports that are positive for ureteric stones. This was attributable to the lack of feature extraction tools tailored for analysing radiology text, incompleteness of the medical lexicon database and heterogeneity of unstructured reports. Improvements in these areas will help improve data extraction accuracy.

A systematic review of natural language processing applied to radiology reports

“…Ten studies use NLP to create specific cohorts for research purposes and six reported the performance of their tools. Out of these papers, the majority (n = 8) created cohorts for specific medical conditions including fatty liver disease [92,93] hepatocellular cancer [94], ureteric stones [95], vertebral fracture [96], traumatic brain injury [97,98], and leptomeningeal disease secondary to metastatic breast cancer [99]. Five papers identified cohorts focused on particular radiology findings including ground glass opacities (GGO) [100], cerebral microbleeds (CMB) [101], pulmonary nodules [102,103], changes in the spine correlated to back pain [1] and identifying radiological evidence of people having suffered a fall.…”

Artificial intelligence in endourology: emerging technology for individualized care

“…NLP has also been applied to radiology reports within the EMR. To identify ureteral stones, Li and Elliot [58] applied NLP algorithms to 1874 manually reviewed noncontrast CT reports. The algorithm identified ureteral stones with 85% accuracy.…”

The Ascent of Artificial Intelligence in Endourology: a Systematic Review Over the Last 2 Decades

“…The accuracy of NLP was 85% with a sensitivity and specificity of 66% and 95% respectively. The low sensitivity and high specificity were due to the lack of feature extraction tools tailored for analyzing radiology text, the incompleteness of the medical lexicon database, and the heterogeneity of unstructured reports [58]. Chen et al used ML methods to study the risk factors (hypertension, increased protein content in stones, decreased calcium oxalate supersaturation, and old age) causing renal stones > 20 mm using demographic variables, 24-h urine profile, and stone profile data of 277 patients.…”