The impact of different strategies to handle missing data on both precision and bias in a drug safety study: a multidatabase multinational population-based cohort study

Abstract

BackgroundMissing data are often an issue in electronic medical records (EMRs) research. However, there are many ways that people deal with missing data in drug safety studies.AimTo compare the risk estimates resulting from different strategies for the handling of missing data in the study of venous thromboembolism (VTE) risk associated with antiosteoporotic medications (AOM).MethodsNew users of AOM (alendronic acid, other bisphosphonates, strontium ranelate, selective estrogen receptor modulators, teriparatide, or denosumab) aged ≥50 years during 1998–2014 were identified in two Spanish (the Base de datos para la Investigación Farmacoepidemiológica en Atención Primaria [BIFAP] and EpiChron cohort) and one UK (Clinical Practice Research Datalink [CPRD]) EMR. Hazard ratios (HRs) according to AOM (with alendronic acid as reference) were calculated adjusting for VTE risk factors, body mass index (that was missing in 61% of patients included in the three databases), and smoking (that was missing in 23% of patients) in the year of AOM therapy initiation. HRs and standard errors obtained using cross-sectional multiple imputation (MI) (reference method) were compared to complete case (CC) analysis – using only patients with complete data – and longitudinal MI – adding to the cross-sectional MI model the body mass index/smoking values as recorded in the year before and after therapy initiation.ResultsOverall, 422/95,057 (0.4%), 19/12,688 (0.1%), and 2,051/161,202 (1.3%) VTE cases/participants were seen in BIFAP, EpiChron, and CPRD, respectively. HRs moved from 100.00% underestimation to 40.31% overestimation in CC compared with cross-sectional MI, while longitudinal MI methods provided similar risk estimates compared with cross-sectional MI. Precision for HR improved in cross-sectional MI versus CC by up to 160.28%, while longitudinal MI improved precision (compared with cross-sectional) only minimally (up to 0.80%).ConclusionCC may substantially affect relative risk estimation in EMR-based drug safety studies, since missing data are not often completely at random. Little improvement was seen in these data in terms of power with the inclusion of longitudinal MI compared with cross-sectional MI. The strategy for handling missing data in drug safety studies can have a large impact on both risk estimates and precision.

Routine Health Outcome Measurement: Development, Design, and Implementation of the Hand and Wrist Cohort

“…[33][34][35][36] In the literature, many statistical analyses and simulation articles have indicated that either multiple imputation techniques or analyses that account for missing data are superior to complete case analyses. [33][34][35][36][37] However, we noticed that such techniques are counterintuitive to many readers. Consequently, we have frequently been asked by journal reviewers to report complete cases, despite literature advising otherwise.…”

A multi-step approach to managing missing data in time and patient variant electronic health records

“…In summarizing the use of EHR data to develop risk prediction models, Goldstein et al [ 9 ] found that only 58 of the 90 studies evaluated addressed missing data prior to analysis. The simplest approaches toward managing missing values involve selecting subsets of the data that contain complete information [ 11 , 12 ], and using stratified mean imputation used to fill-in missing values [ 13 ]. Others have designed functions to interpolate longitudinal variables with limited individual-level variability that are typically not dependent on other covariates [ 14 ].…”

“…Simpler approaches toward EHR imputation must consider whether missing values are missing completely at random (MCAR), missing at random (MAR), or missing not at random (MNAR) [ 14 ]. Conditional imputation methods may be used to account for these dependencies, most effectively if missing data are MAR [ 10 , 12 , 15 ]. While they may improve completeness and predictive precision, these methods may be computationally intensive when applied to large-scale EHR data with significant amounts of missing values.…”

A systematic review of how missing data are handled and reported in multi‐database pharmacoepidemiologic studies

“…8,9 For example, one study showed that risk estimates of venous thromboembolism associated with anti-osteoporotic medications were substantially affected by the use of different strategies for the handling of missing data, leading to differences in the direction of treatment effect estimates. 8 Missing data can arise at several stages within a multi-database pharmacoepidemiologic study. Like in a single database study, data may not be recorded at the stage of data entry into the database.…”

“…14 Methods to account for sporadically missing data, such as multiple imputation (MI) and inverse probability weighting, are widely known. 8,15 To handle systematically missing data, a practical approach is to exclude the missing variable from the analyses or exclude an entire database. 8 A recently proposed alternative is multi-level MI (MLMI), which can account for both sporadically and systematically missing data.…”