Using text analysis software to identify determinants of inappropriate clinical question reporting and diagnostic procedure referrals in Reggio Emilia, Italy

Abstract

Background Inappropriate prescribing of diagnostic procedures leads to overdiagnosis, overtreatment and resource waste in healthcare systems. Effective strategies to measure and to overcome inappropriateness are essential to increasing the value and sustainability of care. We aimed to describe the determinants of inappropriate reporting of the clinical question and of inappropriate imaging and endoscopy referrals through an analysis of general practitioners’ (GP) referral forms in the province of Reggio Emilia, Italy. Methods A clinical audit was conducted on routinely collected referral forms of all GPs of Reggio Emilia province. All prescriptions for gastroscopy, colonoscopy, neurological and musculoskeletal computerised tomography (CT) and magnetic resonance imaging (MRI) from 2012 to 2017 were included. The appropriateness of referral forms was assessed using Clinika VAP software, which combines semantic analysis of clinical questions and available metadata. Local protocols agreed on by all physicians defined criteria of appropriateness. Two multilevel logistic models were used to identify multiple predictors of inappropriateness of referral forms and to analyse variability among GPs, primary care subdistricts and healthcare districts. Results Overall, 37% of referral forms were classified as inappropriate, gastroscopy and CT showed higher proportions of inappropriate referrals compared to colonoscopy and MRI. Inappropriateness increased with patient age for CT and MRI; for gastroscopy, it was lower for patients aged 65–84 compared to those younger, and for colonoscopy, it was higher for older patients. Fee exemptions were associated with inappropriateness in MRI referral forms. The effect of GPs’ practice organization was consistent across all tests, showing higher inappropriateness for primary care medical networks than in primary care medical groups. Male GPs were associated with inappropriateness in endoscopy, and older GPs were associated with inappropriateness in musculoskeletal CT. While there was moderate variability in the inappropriate prescribing among GPs, there was not among the healthcare districts or primary care subdistricts. Conclusions Routinely collected data and IT tools can be useful to identify and monitor diagnostic procedures at high risk of inappropriate prescribing. Assessing determinants of inappropriate referral makes it possible to tailor educational and organizational interventions to those who need them.

The natural language processing of radiology requests and reports of chest imaging: Comparing five transformer models’ multilabel classification and a proof-of-concept study

“…Venturelli et al assessed the appropriateness of referrals for imaging and other diagnostic procedures by analysing the requests’ content using a commercial software package. 41 Their study’s focus was on classification results (appropriate vs. not appropriate) and not the applied method’s performance, which impeded comparison with our study. However, the study demonstrated the feasibility of using requests for assessing appropriateness and is an excellent example of a future application for transformer-based NLP.…”

CT and MRI imaging in Sweden: retrospective appropriateness analysis of large referral samples

“…This study supports these concepts. Other studies suggest and explore new techniques, such as text analysis software and other types of automated language processing, in evaluations [ 18 – 20 ]. This could be a way forward in the future.…”

Justification of CT practices across Europe: results of a survey of national competent authorities and radiology societies

“…While responses may have been influenced by differing interpretations of the question posed as to whether this related to real life practice or to the existence of regulation, it is clear that advance justification is not practically being performed for all CT examinations, given the volume of respondents (47%) who reported that advance justification was performed sometimes or mostly. Although a minority (n = 4) stated that referrals were justified at the point of referral, this could have significant implications for the appropriate use of CT for patients without radiology practitioner oversight, particularly when one considers the poor usage and knowledge of referral guidelines already cited by others [6,8,10,17]. A previous HERCA report [18] following an inspection week in 2016 reported broadly similar findings, with as many as 26% of facilities not performing a satisfactory evaluation of the referral before the examinations were performed and even more not rejecting unjustified examinations (31%) or fully proving that the examinations were authorised by the radiological practitioner (35%).…”

“…Appropriate imaging referrals not only reduce the radiation exposure of the population, but importantly also save valuable healthcare resources. However, numerous publications point to a less than ideal level of justification in current practice with national audits reporting up to 39% [5][6][7] of CT examinations not being justified and even higher rates reported across smaller studies [8][9][10][11].…”