Diagnostic accuracy of ICD code versus discharge summary-based query for endocarditis cohort identification

Abstract

Studies of infective endocarditis (IE) have relied on International Classification of Disease (ICD) codes to identify cases, a method vulnerable to misclassification. Clinical narrative data could offer greater accuracy and richness to cohort identification. We evaluated two algorithms: a standard query of ICD-9/10 billing codes, with or without procedure codes for echocardiogram and a text query of discharge summaries (DS) that selected on the term “endocarditis” in fields headed by “Discharge Diagnosis” or “Admission Diagnosis” or similar. Further coding extracted valve involved and organism responsible if present. All cases were chart reviewed using pre-specified criteria. Positive predictive value (PPV), sensitivity and specificity were calculated. The ICD-based query identified 612 individuals from July 2015 to July 2019 who had a hospital billing code for infective endocarditis; of these, 534 had an echocardiogram. The DS query identified 387 cases. PPV for the DS query was 84.5% (95% CI 80.6%, 87.8%) compared with 72.4% (95% CI 68.7%, 75.8%) for ICD only ( P < .001) and 75.8% (95% CI 72.0%, 79.3%) for ICD + echo queries ( P = .002). Sensitivity was 75.9% for DS query and 86.8% to 93.4% for ICD queries ( P < .02 for these comparisons). Specificity was high for all queries >94%. The DS query also yielded valve data (prosthetic, tricuspid, aortic, etc) in 60% and microbiologic agent in 73% of identified cases with an accuracy of 94% and 90%, respectively when assessed by chart review. Compared with ICD-based queries, text-based queries of discharge summaries have the potential to improve precision of IE case ascertainment and extract key clinical variables.

Deep Learning Analysis of Polish Electronic Health Records for Diagnosis Prediction in Patients with Cardiovascular Diseases

“…For example, AI/NLP-based systems can be used to verify potential discrepancies between EHR-derived original ICD codes manually entered by clinicians and automatically generated ICD codes. Inappropriate diagnostic codes are being reported in an increasing number of publications including cases of stroke [ 40 ], myocardial infarction [ 41 ], and endocarditis [ 42 ]. Tremendous discrepancies were also reported in the ambulatory care in documenting ICD-10 codes for six standardized clinical case scenarios.…”

Lead Extraction and Mortality Among Patients With Cardiac Implanted Electronic Device Infection

“…Such analyses are subject to the sensitivity and specificity of the coding for comorbidities and events. Prior analyses of diagnostic codes for infective endocarditis have identified that they have moderate sensitivity but high specificity (>94%) . As with any observational study, residual measured and unmeasured confounding may have influenced the outcomes despite adjusted modeling.…”

Correlates of 90-Day Mortality Among People Who Do and Do Not Inject Drugs With Infective Endocarditis in Seattle, Washington

“…Although the specificity of our string-searching and pattern-matching algorithm was high, and cases were further adjudicated by a member of the study team, the sensitivity of our algorithm was only 76%, and it is likely we missed cases where IE was not included in the list of discharge diagnoses, was misspelled, or when the discharge summary was formatted in a nonconventional way. Nevertheless, a prior analysis from our group suggested that these aberrations were sporadic in nature and unlikely to be associated with significant selection bias [ 18 ]. In this study, PWID status was collected via chart review only, and therefore may be subject to misclassification, particularly in the absence of clinical documentation.…”

“…Cases of IE were identified by applying a string-searching and pattern-matching text query for endocarditis in the diagnoses field of discharge summaries within the clinical data repository that comprises electronic health records from both hospitals. This method of case identification has been previously published, showing a sensitivity of 76%, a specificity of 98%, and positive predictive value of 85% for identifying cases of IE [ 18 ]. Using a standardized data collection form (REDCap) [ 19 , 20 ], all cases were subsequently confirmed by review of the medical record, which was performed by a member of the study team.…”