Handling missing values in the analysis of between-hospital differences in ordinal and dichotomous outcomes: a simulation study

Abstract

Missing data are frequently encountered in registries that are used to compare performance across hospitals. The most appropriate method for handling missing data when analysing differences in outcomes between hospitals is unclear. We aimed to compare methods for handling missing data when comparing hospitals on ordinal and dichotomous outcomes. We performed a simulation study using data came from the MR CLEAN registry, a prospective cohort study in 17 hospitals performing endovascular therapy for ischemic stroke in the Netherlands. The investigated methods for handling missing data, both case-mix adjustment variables and outcomes, were complete case analysis (CCA), single imputation, multiple imputation, single imputation with deletion of imputed outcomes and multiple imputation with deletion of imputed outcomes. Data were generated as missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR) in three scenarios: (1) 10% missing data in case-mix and outcome; (2) 40% missing data in case-mix and outcome; and (3) 40% missing data in case-mix and outcome with varying degree of missing data among hospitals. Validity and reliability of the methods were compared on the mean squared error (MSE, a summary measure combining bias and precision) relative to the centre effect estimates from the complete reference dataset. For both the ordinal outcome (i.e. the modified Rankin scale) and a common dichotomized version thereof, the MSE of all methods was on average lowest under MCAR and with fewer missing data, and highest with more missing data and under MNAR. The ′ multiple imputation, then deletion′ method had the lowest MSE for both outcomes under all simulated patterns of missing data. Thus, when estimating centre effects on ordinal and dichotomous outcomes in the presence of missing data, the least biased and most precise method to handle these missing data is ′ multiple imputation, then deletion′.