Disease misclassification in electronic healthcare database studies: Deriving validity indices—A contribution from the ADVANCE project

Abstract

There is a strong and continuously growing interest in using large electronic healthcare databases to study health outcomes and the effects of pharmaceutical products. However, concerns regarding disease misclassification (i.e. classification errors of the disease status) and its impact on the study results are legitimate. Validation is therefore increasingly recognized as an essential component of database research. In this work, we elucidate the interrelations between the true prevalence of a disease in a database population (i.e. prevalence assuming no disease misclassification), the observed prevalence subject to disease misclassification, and the most common validity indices: sensitivity, specificity, positive and negative predictive value. Based on this, we obtained analytical expressions to derive all the validity indices and true prevalence from the observed prevalence and any combination of two other parameters. The analytical expressions can be used for various purposes. Most notably, they can be used to obtain an estimate of the observed prevalence adjusted for outcome misclassification from any combination of two validity indices and to derive validity indices from each other which would otherwise be difficult to obtain. To allow researchers to easily use the analytical expressions, we additionally developed a user-friendly and freely available web-application.

Validation of an EMR algorithm to measure the prevalence of ADHD in the Canadian Primary Care Sentinel Surveillance Network (CPCSSN)

“…From these results, it was determined that the ADHD algorithm has a PPV of 95.9% (95% CI [92.6, 98.0]) and an NPV of 96.3% (95% CI [93.2, 98.3]). Using the analytic expressions developed by Bollaerts et al, the derived sensitivity is 19.9% (95% CI [15.9, 25.6]) and the derived specificity is 100.0% (95% CI [99.9, 100.0]) [ 19 ].…”

“…The analysis was performed in SAS statistical software (version 9.6®). The online application developed by Bollaerts et al was used as the analytical tool to calculated adjusted sensitivity and specificity and estimate true prevalence values [19].…”

“…To obtain an estimate of the true prevalence in the database population the observed prevalence was adjusted for misclassification using the validity indices (NPV and PPV) determined by the validation of the chart algorithm. The analytical expressions used to calculate the true prevalence were recently published in a study by Bollaerts et al, and are available as a user-friend web application [ 19 ].…”

“…While sensitivity and specificity could not be directly measured, as the validation sample was not a random selection of patients, we could get direct measures of the positive predictive value (PPV) and the negative predictive value (NPV) for our EMR algorithm. The sensitivity and specificity of the algorithm were derived using analytical expressions based on the Rogan-Gladen estimator [19,20]. The positive predictive value (PPV) of the algorithm was calculated by determining the number of patients classified as having ADHD by both the algorithm and the chart abstractor, out of the all the patients classified as having ADHD by the algorithm.…”

Outcome misclassification: Impact, usual practice in pharmacoepidemiology database studies and an online aid to correct biased estimates of risk ratio or cumulative incidence

“…A “gold standard” comparator may not be available, or direct validation of an outcome may not be feasible. In addition, measurement of sensitivity may not be feasible if the outcome is rare, although it can be calculated if negative predictive value and PPV can be measured 13 . However, caution should be used in applying the PPVs from populations other than the study population as those that differ in terms of disease prevalence may have different PPVs 14 …”