Automated chart review utilizing natural language processing algorithm for asthma predictive index

Abstract

BackgroundThus far, no algorithms have been developed to automatically extract patients who meet Asthma Predictive Index (API) criteria from the Electronic health records (EHR) yet. Our objective is to develop and validate a natural language processing (NLP) algorithm to identify patients that meet API criteria.MethodsThis is a cross-sectional study nested in a birth cohort study in Olmsted County, MN. Asthma status ascertained by manual chart review based on API criteria served as gold standard. NLP-API was developed on a training cohort (n = 87) and validated on a test cohort (n = 427). Criterion validity was measured by sensitivity, specificity, positive predictive value and negative predictive value of the NLP algorithm against manual chart review for asthma status. Construct validity was determined by associations of asthma status defined by NLP-API with known risk factors for asthma.ResultsAmong the eligible 427 subjects of the test cohort, 48% were males and 74% were White. Median age was 5.3 years (interquartile range 3.6–6.8). 35 (8%) had a history of asthma by NLP-API vs. 36 (8%) by abstractor with 31 by both approaches. NLP-API predicted asthma status with sensitivity 86%, specificity 98%, positive predictive value 88%, negative predictive value 98%. Asthma status by both NLP and manual chart review were significantly associated with the known asthma risk factors, such as history of allergic rhinitis, eczema, family history of asthma, and maternal history of smoking during pregnancy (p value < 0.05). Maternal smoking [odds ratio: 4.4, 95% confidence interval 1.8–10.7] was associated with asthma status determined by NLP-API and abstractor, and the effect sizes were similar between the reviews with 4.4 vs 4.2 respectively.ConclusionNLP-API was able to ascertain asthma status in children mining from EHR and has a potential to enhance asthma care and research through population management and large-scale studies when identifying children who meet API criteria.

Expert artificial intelligence-based natural language processing characterises childhood asthma

“…Since most cases of probable asthma became definite asthma over time, both definite and probable asthma were considered as PAC positive 19. Although the API was originally developed to predict asthma among preschoolers, the National Asthma Education and Prevention Program recommended it for identification of asthmatic children for timely asthma treatment (table 1-2) 14 15. We previously reported the details for the development and validation of both NLP algorithms14 15 with a great performance (sensitivity, specificity, positive predictive value and negative predictive value: 97%, 95%, 90% and 98% for NLP-PAC, and 86%, 98%, 88% and 98% for NLP-API).…”

“…Although the API was originally developed to predict asthma among preschoolers, the National Asthma Education and Prevention Program recommended it for identification of asthmatic children for timely asthma treatment (table 1-2) 14 15. We previously reported the details for the development and validation of both NLP algorithms14 15 with a great performance (sensitivity, specificity, positive predictive value and negative predictive value: 97%, 95%, 90% and 98% for NLP-PAC, and 86%, 98%, 88% and 98% for NLP-API). Briefly, both NLP algorithms had the sequential process to determine positivity for asthma criteria: (1) the text extraction that searches evidence concepts for asthma in EHRs, (2) processing the extracted concepts based on rules for asthma criteria and (3) categorising asthma status accordingly.…”

“…Given the growing deployment of electronic health records (EHRs) systems enabling large practice-based longitudinal data mining, advancement in artificial intelligence (AI) approaches such as natural language processing (NLP; expert AI) may potentially enable us to address these challenges as it can extract, process and classify free-text data from EHRs 12–15. For example, we recently developed and validated NLP algorithms for two existing retrospective criteria for childhood asthma (NLP algorithms for Predetermined Asthma Criteria (NLP-PAC) and NLP algorithms for Asthma Predictive Index (NLP-API)) 14 15. The performance of individual NLP algorithm determining asthma status based on comprehensive EHRs including free text was almost close to that by humans (eg, 97% sensitivity and 95% specificity for NLP-PAC) 14.…”

Measuring problem prescription opioid use among patients receiving long-term opioid analgesic treatment: development and evaluation of an algorithm for use in EHR and claims data

“…Gold standard data generated using the algorithm developed in this study were to be combined with gold standard data on opioid-related overdoses developed in a companion study and used to investigate the incidence and epidemiology of problem opioid use and opioid-related overdose and death 23 in a very large patient cohort combining data from Kaiser Permanente Northwest (KPNW), KPWA, Optum, and Tennessee Medicaid. As such, this study also contributes to an emerging literature on automated methods to determine patient phenotypes or case status in "big" healthcare data to support clinical, epidemiological and surveillance research without the need for expensive, sampleconstraining manual chart review [24][25][26] .…”

Machine Learning in Relation to Emergency Medicine Clinical and Operational Scenarios: An Overview

“…3 Several other examples of using NLP for detection and prediction of disease and adverse events are reported in the literature. [34][35][36][37][38]…”