Validation of Pediatric Diabetes Case Identification Approaches for Diagnosed Cases by Using Information in the Electronic Health Records of a Large Integrated Managed Health Care Organization

Abstract

We explored the utility of different algorithms for diabetes case identification by using electronic health records. Inpatient and outpatient diagnosis codes, as well as data on laboratory results and dispensing of antidiabetic medications were extracted from electronic health records of Kaiser Permanente Southern California members who were less than 20 years of age in 2009. Diabetes cases were ascertained by using the SEARCH for Diabetes in Youth Study protocol and comprised the "gold standard." Sensitivity, specificity, positive and negative predictive values, accuracy, and the area under the receiver operating characteristic curve (AUC) were compared in 1,000 bootstrapped samples. Based on data from 792,992 youth, of whom 1,568 had diabetes (77.2%, type 1 diabetes; 22.2%, type 2 diabetes; 0.6%, other), case identification accuracy was highest in 75% of bootstrapped samples for those who had 1 or more outpatient diabetes diagnoses or 1 or more insulin prescriptions (sensitivity, 95.9%; positive predictive value, 95.5%; AUC, 97.9%) and in 25% of samples for those who had 2 or more outpatient diabetes diagnoses and 1 or more antidiabetic medications (sensitivity, 92.4%; positive predictive value, 98.4%; AUC, 96.2%). Having 1 or more outpatient type 1 diabetes diagnoses (International Classification of Diseases, Ninth Revision, Clinical Modification, code 250.x1 or 250.x3) had the highest accuracy (94.4%) and AUC (94.1%) for type 1 diabetes; the absence of type 1 diabetes diagnosis had the highest accuracy (93.8%) and AUC (93.6%) for identifying type 2 diabetes. Information in the electronic health records from managed health care organizations provides an efficient and cost-effective source of data for childhood diabetes surveillance.

Use of administrative and electronic health record data for development of automated algorithms for childhood diabetes case ascertainment and type classification: the SEARCH for Diabetes in Youth Study

“…There were minimal differences in performance of type 1 algorithms across age and racial/ethnic groups. These results are consistent with previous studies that have reported that youth with type 1 diabetes are easier to accurately identify relative to youth with type 2 diabetes . Interestingly, the performance of type 1 algorithms was not improved with addition of medication and laboratory data.…”

“…As for fasting/random glucose, the performance differed significantly by age groups. The PPV of the fasting/random glucose criterion (45.9%) was lower than that in the recent analysis by Lawrence et al (>60%) ; however, only 6–12 months of laboratory data were used in their study. In our study, the PPV increased to a comparable level of 63.2% when 1 yr of data was used while the sensitivity remained similar (data not shown).…”

“…In fact, the PPV increased only from 16.6 to 31.0% in children <10 yr age group using 1 yr of data. Our population had a proportion of children < 5 yr twice that of the sample used in the study by Lawrence et al The findings of this study suggest that glucose criterion may not accurately ascertain childhood diabetes cases in children <10 yr. The reasons may involve the fact that younger youth visit health system more frequently and a qualified glucose test result can frequently be triggered by conditions other than diabetes.…”

Bias from outcome misclassification in immunization schedule safety research

“…We identified validation studies of EHR‐based algorithms for 2 outcomes of interest: asthma and Type 1 diabetes . The best performing algorithms had sensitivity > 95% and specificity > 99% . For each simulation and RR, we made 4 copies of the 1000 replicated datasets and tested 2 scenarios of non‐differential and 2 scenarios of differential outcome misclassification based on these levels (Table , scenarios A–D).…”

“…Traditional QBA formulas for outcome misclassification apply sensitivity and specificity estimates to a study's observed relative risk. Previous studies have reported the sensitivity and specificity for several chronic outcomes of interest in immunization schedule safety studies; these measures could be used as bias parameters in QBA.…”