Evaluation of an Algorithm for Identifying Ocular Conditions in Electronic Health Record Data

Abstract

IMPORTANCE For research involving big data, researchers must accurately identify patients with ocular diseases or phenotypes of interest. Reliance on administrative billing codes alone for this purpose is limiting. OBJECTIVE To develop a method to accurately identify the presence or absence of ocular conditions of interest using electronic health record (EHR) data. DESIGN, SETTING, AND PARTICIPANTS This study is a retrospective analysis of the EHR data of patients (n = 122 339) in the Sight Outcomes Research Collaborative Ophthalmology Data Repository who received eye care at participating academic medical centers between August 1, 2012, and August 31, 2017. An algorithm that searches structured and unstructured (free-text) EHR data for conditions of interest was developed and then tested to determine how well it could detect the presence or absence of exfoliation syndrome (XFS). The algorithm was trained to search for evidence of XFS among a sample of patients with and without XFS (n = 200) by reviewing International Classification of Diseases, Ninth Revision or International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-9 or ICD-10) billing codes, the patient's problem list, and text within the ocular examination section and unstructured (free-text) data in the EHR. The likelihood that each patient had XFS was estimated using logistic least absolute shrinkage and selection operator (LASSO) regression. The EHR data of all patients were run through the algorithm to generate an XFS probability score for each patient. The algorithm was validated with review of EHRs by glaucoma specialists. MAIN OUTCOMES AND MEASURES Positive predictive value (PPV) and negative predictive value (NPV) of the algorithm were computed as the proportion of patients correctly classified with XFS or without XFS. RESULTS This study included 122 339 patients, with a mean (SD) age of 52.4 (25.1) years. Of these patients, 69 002 (56.4%) were female and 99 579 (81.4%) were white. The algorithm assigned a less than 10% probability of XFS for 121 085 patients (99.0%) as well as an XFS probability score of more than 75% for 543 patients (0.4%), more than 90% for 353 patients (0.3%), and more than 99% for 83 patients (0.07%). Validated by glaucoma specialists, the algorithm had a PPV of 95.0% (95% CI, 89.5%-97.7%) and an NPV of 100% (95% CI, 91.2%-100%). When there was ICD-9 or ICD-10 billing code documentation of XFS, in 86% or 96% of the records, respectively, evidence of XFS was also recorded elsewhere in the EHR. Conversely, when there was clinical examination or free-text evidence of XFS, it was documented with ICD-9 codes only approximately 40% of the time and even less often with ICD-10 codes. CONCLUSIONS AND RELEVANCE The algorithm developed, tested, and validated in this study appears to be better at identifying the presence or absence of XFS in EHR data than the conventional approach of assessing only billing codes; such an algorithm may enhance the ability of investigators to use EHR data to stud...

How Have Intravitreal Anti-VEGF and Dexamethasone Implant Been Used in Italy? A Multiregional, Population-Based Study in the Years 2010–2016

“…(2) DME (narrow definition): DME was identified based on (a) a diabetes diagnosis in a hospital discharge record or (b) at least two pharmacy claims for antidiabetic drugs and a diagnosis of DME in a hospital discharge record; (3) DME (broad definition): DME was defined based on (a) a diabetes diagnosis in a hospital discharge record or (b) at least two pharmacy claims for antidiabetic drugs. e diagnostic and predictive accuracy of each claims-based algorithm was evaluated by calculating the sensitivity, specificity, overall accuracy, positive predictive value (PPV), and negative predictive value (NPV) with respective 95% CIs [12].…”

Text Processing for Detection of Fungal Ocular Involvement in Critical Care Patients: Cross-Sectional Study

“…NLP has been used to abstract findings from radiology and pathology reports [ 40 - 44 ], for identifying phenotypes from narrative notes [ 45 - 48 ], and specifically for ophthalmic data extraction such as visual acuity and surgical complications [ 20 - 22 ]. NLP has been shown to enhance case detection compared to structured diagnosis codes alone, for example, for identifying cases of pseudoexfoliation syndrome [ 26 ] and herpes zoster ophthalmicus [ 25 ]. Structured diagnosis codes have known limitations such as incomplete or inaccurate coding, insufficient granularity, and the fact that clinicians may not code for every condition at every encounter [ 29 ].…”

“…NLP has been applied to extract data on visual acuity [ 20 ] and intracameral antibiotic injections and posterior capsular rupture [ 21 ]. It has also been used to extract surgical laterality and intraocular lens implant power and model information [ 22 ], glaucoma-related characteristics [ 23 ], measurements of epithelial defects and stromal infiltrates in microbial keratitis [ 24 ], as well as identification of herpes zoster ophthalmicus [ 25 ] and pseudoexfoliation [ 26 ].…”

Development of pedotransfer functions by machine learning for prediction of soil electrical conductivity and organic carbon content

“…The approach has theoretical similarity with machine learning methods that use variants of GLMM and log-link functions derived from logistic models (Benke, 2019;Hengl et al, 2018;Liao et al, 2015;McNeill et al, 2018;Stein et al, 2019). An advantage of GLMM with REML is that it is not a predictive 'black box' model, but rather provides added explanatory behaviour by exposing linear or non-linear relationships between covariates and possible interactions, and therefore more information on the nature of the actual physical processes driving predictions.…”

“…Some strengths and weaknesses of machine learning approaches in general have been discussed in the literature (see also Castelvecchi, 2016;Liao et al, 2015;McNeill et al, 2018;Nilsson, 2014;Russell and Norvig, 2016;Stein et al, 2019). Physical models have advantages aside from being more explanatory than non-parametric machine learning algorithms in that they provide a theoretical basis for understanding soil processes, they support planning new research strategies, and are more robust, reliable and less sensitive to noise and artefacts in limited datasets.…”