Validation of an algorithm to identify children with biopsy-proven celiac disease from within health administrative data: An assessment of health services utilization patterns in Ontario, Canada

Abstract

ImportanceCeliac disease (CD) is a common pediatric illness, and awareness of gluten-related disorders including CD is growing. Health administrative data represents a unique opportunity to conduct population-based surveillance of this chronic condition and assess the impact of caring for children with CD on the health system.ObjectiveThe objective of the study was to validate an algorithm based on health administrative data diagnostic codes to accurately identify children with biopsy-proven CD. We also evaluated trends over time in the use of health services related to CD by children in Ontario, Canada.Study design and settingWe conducted a retrospective cohort study and validation study of population-based health administrative data in Ontario, Canada. All cases of biopsy-proven CD diagnosed 2005–2011 in Ottawa were identified through chart review from a large pediatric health care center, and linked to the Ontario health administrative data to serve as positive reference standard. All other children living within Ottawa served as the negative reference standard. Case-identifying algorithms based on outpatient physician visits with associated ICD-9 code for CD plus endoscopy billing code were constructed and tested. Sensitivity, specificity, PPV and NPV were tested for each algorithm (with 95% CI). Poisson regression, adjusting for sex and age at diagnosis, was used to explore the trend in outpatient visits associated with a CD diagnostic code from 1995–2011.ResultsThe best algorithm to identify CD consisted of an endoscopy billing claim follow by 1 or more adult or pediatric gastroenterologist encounters after the endoscopic procedure. The sensitivity, specificity, PPV, and NPV for the algorithm were: 70.4% (95% CI 61.1–78.4%), >99.9% (95% CI >99.9->99.9%), 53.3% (95% CI 45.1–61.4%) and >99.9% (95% CI >99.9->99.9%) respectively. It identified 1289 suspected CD cases from Ontario-wide administrative data. There was a 9% annual increase in the use of this combination of CD-associated diagnostic codes in physician billing data (RR 1.09, 95% CI 1.07–1.10, P<0.001).ConclusionsWith its current structure and variables Ontario health administrative data is not suitable in identifying incident pediatric CD cases. The tested algorithms suffer from poor sensitivity and/or poor PPV, which increase the risk of case misclassification that could lead to biased estimation of CD incidence rate. This study reinforced the importance of validating the codes used to identify cohorts or outcomes when conducting research using health administrative data.

Validation study of health administrative data algorithms to identify individuals experiencing homelessness and estimate population prevalence of homelessness in Ontario, Canada

“…Due to the low prevalence (<5%) of exclusively housed individuals in this cohort, an additional group of adults presumed housed (the ‘general population sample’) was randomly selected from the ICES Registered Persons Database (RPDB), which includes all individuals eligible for Ontario health coverage. A similar approach was used in previous validation studies 31 32. To ensure our general population sample had a high likelihood of being housed, we deemed individuals eligible if they were not part of the HHiT study, resided in Toronto or Ottawa throughout the study period and did not reside in a postal code associated with shelter services.…”

Development of an administrative definition for celiac disease

“…A previous study examining the role of ICD-9-CM codes in predicting the diagnosis of celiac disease demonstrated a very low PPV when this was used in isolation [18]. More recently, in a pediatric study from Canada, there was excellent specificity (> 99%) but only moderate sensitivity (70.4%) when a claim for a gastroscopy and 1 or more outpatient claims by a pediatric or adult gastroenterologist was made [19]. This was a pediatric study using intestinal biopsy to validate the case definition and included 125 biopsy proven celiac cases.…”

Systematic review of validated case definitions to identify hypertensive disorders of pregnancy in administrative healthcare databases

“…14 Unfortunately, we were unable to fully interpret the reliability of validation statistics reported across studies, as reporting of participant characteristics was poor, and most studies failed to provide adequate descriptions of HDP severity, or report on additional clinical factors associated with HDP severity (eg, participants who experienced a preterm birth), for their validation cohorts. 57 Choosing the best CFD(s) for HDP CFDs should always be chosen in light of several important considerations, including (1) applicability to available data sources, [58][59][60] (2) validation study design and participant characteristics, (3) methodological rigour and (4) assessments of bias and reporting quality for the studies from which they were derived. The relative importance of high SN, SP, NPV and/or PPV must also be considered, and considerations prioritised based on study objectives.…”