External validation of a claims-based model to predict left ventricular ejection fraction class in patients with heart failure

Abstract

Background Ejection fraction (EF) is an important prognostic factor in heart failure (HF), but administrative claims databases lack information on EF. We previously developed a model to predict EF class from Medicare claims. Here, we evaluated the performance of this model in an external validation sample of commercial insurance enrollees. Methods Truven MarketScan claims linked to electronic medical records (EMR) data (IBM Explorys) containing EF measurements were used to identify a cohort of US patients with HF between 01-01-2012 and 10-31-2019. By applying the previously developed model, patients were classified into HF with reduced EF (HFrEF) or preserved EF (HFpEF). EF values recorded in EMR data were used to define gold-standard HFpEF (LVEF ≥45%) and HFrEF (LVEF<45%). Model performance was reported in terms of overall accuracy, positive predicted values (PPV), and sensitivity for HFrEF and HFpEF. Results A total of 7,001 HF patients with an average age of 71 years were identified, 1,700 (24.3%) of whom had HFrEF. An overall accuracy of 0.81 (95% CI: 0.80–0.82) was seen in this external validation sample. For HFpEF, the model had sensitivity of 0.96 (95%CI, 0.95–0.97) and PPV of 0.81 (95% CI, 0.81–0.82); while for HFrEF, the sensitivity was 0.32 (95%CI, 0.30–0.34) and PPV was 0.73 (95%CI, 0.69–0.76). These results were consistent with what was previously published in US Medicare claims data. Conclusions The successful validation of the Medicare claims-based model provides evidence that this model may be used to identify patient subgroups with specific EF class in commercial claims databases as well.

Association of Medicare-Medicaid Dual Eligibility and Race and Ethnicity With Ischemic Stroke Severity

“…Third, administrative claims data may have inaccurate coding and potential missing data, although high levels of agreement have been reported between administrative data and electronic medical records. 52 , 53 , 54 Fourth, inpatient claims information for Medicare Advantage (MA) patients are still missing in current Medicare data. Since missing NIHSS scores in MA patients could induce bias in our study, we excluded MA patients.…”

Machine Learning Approaches for Electronic Health Records Phenotyping: A Methodical Review

“…Traditionally, gold-standard labels are annotated by manual review of patient records [37,85,116]. Labels have also been derived from registry data [33], laboratory results [61,112,117], diagnosis codes [30,57,58,118–120], and rule-based algorithms [59,121–123] to enable more rapid development of labeled datasets. The most commonly used methods for classifying a binary phenotype are random forest [26,28,35,37,56,57,60,62,70,81,84,117,119,120,124–126], logistic regression [36,37,57,58,60,67,82,84,93,116,117,119,125,127,128], and support vector machine (SVM) [31,35,37,58,60,81,82,84,92,97,104,116,125,126] (Supplementary Material Table S12 ).…”

“…Nearly one-third of studies used openly available datasets such as datasets released by natural language processing (NLP) competitions and the Medical Information Mart for Intensive Care (MIMIC-III) [55] (Supplementary Material Table S6) . Several studies linked EHR data to administrative claims [39,[56][57][58][59][60][61] or registry databases [45,[62][63][64] to validate the accuracy of an EHR-based phenotyping method. Specimen biorepositories have also been used to demonstrate the accuracy of a derived phenotype in genetic applications, such as a replication of a genome-wide association study [65][66][67][68].…”

“…A majority of articles (n = 18, 31%) developed accurate algorithms to phenotype chronic conditions such as cancer [62,128] and heart failure [61]. Several studies considered rare phenotypes which required methods for the imbalanced class setting [58,106].…”

“…[29] The remaining articles leveraged publicly available data from the Medical Information Mart for Intensive Care (MIMIC-III) database and NLP competitions (Table S6). A small number of studies utilized additional data sources, including administrative claims [30][31][32][33][34][35][36] and registry databases. [37][38][39][40] 94 studies were conducted in the US.…”