Impact of missing data on bias and precision when estimating change in patient-reported outcomes from a clinical registry

Abstract

Background Clinical registries, which capture information about the health and healthcare use of patients with a health condition or treatment, often contain patient-reported outcomes (PROs) that provide insights about the patient’s perspectives on their health. Missing data can affect the value of PRO data for healthcare decision-making. We compared the precision and bias of several missing data methods when estimating longitudinal change in PRO scores. Methods This research conducted analyses of clinical registry data and simulated data. Registry data were from a population-based regional joint replacement registry for Manitoba, Canada; the study cohort consisted of 5631 patients having total knee arthroplasty between 2009 and 2015. PROs were measured using the 12-item Short Form Survey, version 2 (SF-12v2) at pre- and post-operative occasions. The simulation cohort was a subset of 3000 patients from the study cohort with complete PRO information at both pre- and post-operative occasions. Linear mixed-effects models based on complete case analysis (CCA), maximum likelihood (ML) and multiple imputation (MI) without and with an auxiliary variable (MI-Aux) were used to estimate longitudinal change in PRO scores. In the simulated data, bias, root mean squared error (RMSE), and 95% confidence interval (CI) coverage and width were estimated under varying amounts and types of missing data. Results Three thousand two hundred thirty (57.4%) patients in the study cohort had complete data on the SF-12v2 at both occasions. In this cohort, mixed-effects models based on CCA resulted in substantially wider 95% CIs than models based on ML and MI methods. The latter two methods produced similar estimates and 95% CI widths. In the simulation cohort, when 50% of the data were missing, the MI-Aux method, in which a single hypothetical auxiliary variable was strongly correlated (i.e., 0.8) with the outcome, reduced the 95% CI width by up to 14% and bias and RMSE by up to 50 and 45%, respectively, when compared with the MI method. Conclusions Missing data can substantially affect the precision of estimated change in PRO scores from clinical registry data. Inclusion of auxiliary information in MI models can increase precision and reduce bias, but identifying the optimal auxiliary variable(s) may be challenging.

Psychosocial Impact of COVID-19 Nursing Home Restrictions on Visitors of Residents With Cognitive Impairment: A Cross-Sectional Study as Part of the Engaging Remotely in Care (ERiC) Project

“…Another limitation is that there were some randomly distributed missing data. This can lead to bias and reduced precision when analyzing patientreported outcomes (37). Surveys are prone to having missing data although in this case, the number of missing values was low.…”

Comparison of Single and MICE Imputation Methods for Missing Values: A Simulation Study

“…Missing data can occur for various reasons during the data collection process, such as incomplete responses (Pampaka et al, 2016;Barnett et al, 2017), equipment malfunction (Masconi et al, 2015;Gopal et al, 2019) and manual data entry errors (Bhati & Gupta, 2016). Incomplete data is a serious data quality problem since it leads to a reduction of statistical power, bias in parameter estimates, and loss of efficiency in the analytical process (Kaiser, 2014;Ayilara et al, 2019;Hughes et al, 2019). These problems have led to extensive research on developing methods to treat missing data.…”

Comparison of Methods for Processing Missing Values in Large Sample Survey Data

“…The analysis models would be the same as other statistical model with complete data sets. Many studies indicated that the imputation model should contain all variables in the analysis model or any auxiliary variables relating with outcome variables likely to be used in the subsequent analyses [19,23]. For each of 30 simulation data sets and the data set K, taking Y 1 as the dependent variable and the others as the covariates for regression analysis.…”

“…For MI, it can provide unbiased estimates of the regression parameter of interest when the missing data is MAR or MCAR. Recent study found when the missing data is assumed to be MAR or MCAR, CCA was performed well (e.g., unbiased risk difference, 95% coverage) [5], although some papers indicated that the method can result in substantially bias [3,9,19]. The application of SI is the same as CCA and MI, for example, inverse probability weighting (IPW) is typically implemented assuming MAR [20][21].…”