‘Caveat emptor’: the cautionary tale of endocarditis and the potential pitfalls of clinical coding data—an electronic health records study

Abstract

Background Diagnostic codes from electronic health records are widely used to assess patterns of disease. Infective endocarditis is an uncommon but serious infection, with objective diagnostic criteria. Electronic health records have been used to explore the impact of changing guidance on antibiotic prophylaxis for dental procedures on incidence, but limited data on the accuracy of the diagnostic codes exists. Endocarditis was used as a clinically relevant case study to investigate the relationship between clinical cases and diagnostic codes, to understand discrepancies and to improve design of future studies. Methods Electronic health record data from two UK tertiary care centres were linked with data from a prospectively collected clinical endocarditis service database (Leeds Teaching Hospital) or retrospective clinical audit and microbiology laboratory blood culture results (Oxford University Hospitals Trust). The relationship between diagnostic codes for endocarditis and confirmed clinical cases according to the objective Duke criteria was assessed, and impact on estimations of disease incidence and trends. Results In Leeds 2006–2016, 738/1681(44%) admissions containing any endocarditis code represented a definite/possible case, whilst 263/1001(24%) definite/possible endocarditis cases had no endocarditis code assigned. In Oxford 2010–2016, 307/552(56%) reviewed endocarditis-coded admissions represented a clinical case. Diagnostic codes used by most endocarditis studies had good positive predictive value (PPV) but low sensitivity (e.g. I33-primary 82% and 43% respectively); one (I38-secondary) had PPV under 6%. Estimating endocarditis incidence using raw admission data overestimated incidence trends twofold. Removing records with non-specific codes, very short stays and readmissions improved predictive ability. Estimating incidence of streptococcal endocarditis using secondary codes also overestimated increases in incidence over time. Reasons for discrepancies included changes in coding behaviour over time, and coding guidance allowing assignment of a code mentioning ‘endocarditis’ where endocarditis was never mentioned in the clinical notes. Conclusions Commonly used diagnostic codes in studies of endocarditis had good predictive ability. Other apparently plausible codes were poorly predictive. Use of diagnostic codes without examining sensitivity and predictive ability can give inaccurate estimations of incidence and trends. Similar considerations may apply to other diseases. Health record studies require validation of diagnostic codes and careful data curation to minimise risk of serious errors. Electronic supplementary material The online version of this article (10.1186/s12916-019-1390-x) contains supplementary material, which is available to authorized users.

Pulmonary complications observed in patients with infective endocarditis with and without injection drug use: An analysis of the National Inpatient Sample

“…Hospitalizations of adult (�18 years old) patients diagnosed with IE were identified using ICD-10-CM codes and included in the study population, S1 Table in S1 File [16][17][18][19][20]. Patients discharged against medical advice (n = 768, 4%, unweighted) or with unknown disposition (n = 90, <1%, unweighted) were excluded since their course of care and clinical outcomes could not be reliably assessed.…”

Investigation of the impact of the NICE guidelines regarding antibiotic prophylaxis during invasive dental procedures on the incidence of infective endocarditis in England: an electronic health records study

“…The most recent comparable study using English data was published in 2015 [6] and reported an increase in the incidence rate of infective endocarditis following publication of the NICE guidelines. However, when we used different case definitions based on a recent study [17] and different statistical methods which identify the most likely date that trends changed, we found a wide range of likely dates for a change in incidence trends, leading us to conclude that there is no evidence for a direct link with the change in guidance in 2008. Although Criteria C implements the inclusion as reported in the earlier study [6], we found small differences in estimated incidence compared to this publication and found much higher and more stable coverage of secondary ICD-10 codes for organisms than previously reported, despite theoretically using the same underlying HES data.…”

“…Criteria A: At least one of the ICD-10 codes: I33.0, I33.9, I39.0, I39.8, I01.1, B37.6, or T82.6 in any diagnosis field, or I38 in the primary diagnosis field, in any episode in a superspell, where the patient was not discharged alive within 2 days, and excluding any readmissions within 30 days (using the HES patient ID as the patient identifier) Criteria B: ICD-10 code I33.0 in the primary diagnosis field, in any episode in a superspell, where the patient was not discharged alive within 2 days, excluding any readmissions within 30 days (using the HES patient ID as the patient identifier), and excluding elective admissions Criteria C: ICD-10 code I33.0 in the primary diagnosis field, in any episode in a superspell, excluding those with an admission method of "Elective -waiting list" Criteria A and B were shown by Fawcett et al [17] to represent the true number of infective endocarditis cases more accurately than simpler criteria, with Criteria A maximising sensitivity plus positive predictive value (PPV) and Criteria B maximising specificity plus PPV, while Criteria C was that employed by Dayer et al [6], the most prominent prior study based on national HES data.…”

“…A recent study of ICD-10 (International Classification of Diseases, Tenth Revision) diagnosis codes used to represent infective endocarditis cases at two large English hospital trusts [17] concluded that the inclusion criteria for observational studies using electronic health records (EHRs) need to be selected very carefully, as, even when specific diagnostic codes are chosen with care, individual records may still not always represent confirmed clinical cases. To build on this, we conducted a range of analyses using national EHR data on infective endocarditis in England, in particular investigating whether changes in incidence could be detected around the change in NICE guidelines or at other times, and assessing the impact of inclusion criteria and statistical model choice on inferences drawn about timing and types of change.…”

Review of Clinical Research Informatics

“…Taylor et al, use developed phenotypes to identify patterns of comorbidities in eMERGE network institutions [64]. That this work is far from easy is readily demonstrated by the difficulties described in other works, such as the study by Fawcett et al, in the UK [65] and that by Ando et al,in Japan [66]. Nevertheless, independent efforts to phenotype particular conditions still arise and show considerable promise.…”