Combining multiple imputation and bootstrap in the analysis of cost‐effectiveness trial data

Abstract

In healthcare cost-effectiveness analysis, probability distributions are typically skewed and missing data are frequent. Bootstrap and multiple imputation are well-established resampling methods for handling skewed and missing data. However, it is not clear how these techniques should be combined. This paper addresses combining multiple imputation and bootstrap to obtain confidence intervals of the mean difference in outcome for two independent treatment groups. We assessed statistical validity and efficiency of 10 candidate methods and applied these methods to a clinical data set. Single imputation nested in the bootstrap percentile method (with added noise to reflect the uncertainty of the imputation) emerged as the method with the best statistical properties. However, this method can require extensive computation times and the lack of standard software makes this method not accessible for a larger group of researchers. Using a standard unpaired t-test with standard multiple imputation without bootstrap appears to be a robust alternative with acceptable statistical performance for which standard multiple imputation software is available.

Bootstrap inference for multiple imputation under uncongeniality and misspecification

“…However, we expect the intervals to be unnecessarily wide with M ¼ 1 because as shown by equation (6), with one imputation the estimator is subject to a relatively large amount of Monte-Carlo error. This is confirmed by the simulation results of Brand et al 11 ( Figure 1, panel C), which shows that the bootstrap percentile intervals were wider on average with M ¼ 1 compared with M ¼ 5. Moreover, their results suggested that coverage with M ¼ 1 was slightly above the nominal 95% level, which we investigate further in Section 4.…”

“…Brand et al also found that the Boot MI percentile approach worked well in simulations. 11 They investigated it using either M ¼ 5 or M ¼ 1, and among the different combinations of bootstrapping and MI recommended using it with M ¼ 1. Provided the MI point estimator is consistent, we would expect the resulting confidence intervals to have correct coverage under uncongeniality or misspecification.…”

“…10 Lastly, Brand et al investigated six different combinations of MI with bootstrapping in the context of handling skewed data and recommended using percentile bootstrap confidence intervals with single (stochastic) imputation. 11 In this paper, we investigate the properties of the different combinations of MI and bootstrap which have been recommended by these previous papers, giving particular emphasis to their validity under uncongeniality or model misspecification. In Section 2, we review MI, Rubin's combination rules and congeniality.…”

Effectiveness of TOcilizumab in comparison to Prednisone In Rheumatoid Arthritis patients with insufficient response to disease-modifying antirheumatic drugs (TOPIRA): study protocol for a pragmatic trial

“…Differences in statistical significance will be tested using the Cochran-Mantel-Haenszeltest, taking stratification for the center into account. Bootstrapping will be combined with single imputation to account for uncertainty in the costeffectiveness outcomes and missing data in the utility and cost categories [28]. Also cost-effectiveness acceptability planes as well as cost-effectiveness acceptability curves will be presented, thereby graphing the probability that the TCZ strategy is costeffective compared to using the prednisone strategy as a function of willingness to pay.…”

Laparoscopic supracervical hysterectomy compared with second-generation endometrial ablation for heavy menstrual bleeding: the HEALTH RCT

“…The variance surrounding the joint incremental costs and effects was characterised using non-parametric bootstrapping (1000 iterations), with multiple imputation (m = 5) nested within the bootstrap loops. 41…”