Identifying influenza-like illness presentation from unstructured general practice clinical narrative using a text classifier rule-based expert system versus a clinical expert

Abstract

BackgroundWe designed and validated a rule-based expert system to identify influenza like illness (ILI) from routinely recorded general practice clinical narrative to aid a larger retrospective research study into the impact of the 2009 influenza pandemic in New Zealand.MethodsRules were assessed using pattern matching heuristics on routine clinical narrative. The system was trained using data from 623 clinical encounters and validated using a clinical expert as a gold standard against a mutually exclusive set of 901 records.ResultsWe calculated a 98.2 % specificity and 90.2 % sensitivity across an ILI incidence of 12.4 % measured against clinical expert classification. Peak problem list identification of ILI by clinical coding in any month was 9.2 % of all detected ILI presentations. Our system addressed an unusual problem domain for clinical narrative classification; using notational, unstructured, clinician entered information in a community care setting. It performed well compared with other approaches and domains. It has potential applications in real-time surveillance of disease, and in assisted problem list coding for clinicians.ConclusionsOur system identified ILI presentation with sufficient accuracy for use at a population level in the wider research study. The peak coding of 9.2 % illustrated the need for automated coding of unstructured narrative in our study.

Quantifying the incidence and burden of herpes zoster in New Zealand general practice: a retrospective cohort study using a natural language processing software inference algorithm

“…The strength of the NLP algorithm-based method used in our study is improved accuracy compared with the use of clinical codes or a simple keyword search, or review by a single clinical expert. 19 32 This methodology, as opposed to a simple keyword search, is able to identify the context in which pertinent terms are being used in clinical narrative. For example, a keyword may be used by a clinician to express either the presence or absence of the disease, which impacts the specificity and PPV of that approach and ultimately overestimating disease.…”

“… 18 To date this software has shown the ability to analyse service utilisation for H1N1 influenza and childhood respiratory diseases while eliminating reliance on clinical coding. 19 20 …”

Electronic Sentinel Surveillance of Influenza-like Illness

“…Some felt that eILI reporting led to less double handling of data, but others considered it to be an extra piece of work during the limited patient consultation time. In future, a data mining approach which can potentially identify ILI patients from electronic medical records in a Practice Management System could potentially save GP's time [19]. However, the data mining approach would lack the case evaluation by medical staff.…”

A Systematic Review of Natural Language Processing for Knowledge Management in Healthcare

“…Data acquired from an open source such as online platform is low in term of quality and need much more time in cleaning and pre-pressing to get acceptable quality. Another limitation is the sample size of data which tends to influence the accuracy of results [20], [24], [18], [31], [14] some of the studies even having sample sizes less than 1500 to as low as 221. This states that a limited number of observations hampers the performance of models which ultimately results in low accuracy.…”