Inference following multiple imputation for generalized additive models: an investigation of the median p-value rule with applications to the Pulmonary Hypertension Association Registry and Colorado COVID-19 hospitalization data

Abstract

Background Missing data prove troublesome in data analysis; at best they reduce a study’s statistical power and at worst they induce bias in parameter estimates. Multiple imputation via chained equations is a popular technique for dealing with missing data. However, techniques for combining and pooling results from fitted generalized additive models (GAMs) after multiple imputation have not been well explored. Methods We simulated missing data under MCAR, MAR, and MNAR frameworks and utilized random forest and predictive mean matching imputation to investigate a variety of rules for combining GAMs after multiple imputation with binary and normally distributed outcomes. We compared multiple pooling procedures including the “D2” method, the Cauchy combination test, and the median p-value (MPV) rule. The MPV rule involves simply computing and reporting the median p-value across all imputations. Other ad hoc methods such as a mean p-value rule and a single imputation method are investigated. The viability of these methods in pooling results from B-splines is also examined for normal outcomes. An application of these various pooling techniques is then performed on two case studies, one which examines the effect of elevation on a six-minute walk distance (a normal outcome) for patients with pulmonary arterial hypertension, and the other which examines risk factors for intubation in hospitalized COVID-19 patients (a dichotomous outcome). Results In comparison to the results from generalized additive models fit on full datasets, the median p-value rule performs as well as if not better than the other methods examined. In situations where the alternative hypothesis is true, the Cauchy combination test appears overpowered and alternative methods appear underpowered, while the median p-value rule yields results similar to those from analyses of complete data. Conclusions For pooling results after fitting GAMs to multiply imputed datasets, the median p-value is a simple yet useful approach which balances both power to detect important associations and control of Type I errors.

Impact of the COVID‐19 pandemic on chronic disease management and patient reported outcomes in patients with pulmonary hypertension: The Pulmonary Hypertension Association Registry

“…However, if the data were truly missing not at random (MNAR), evidence suggests our approach would be less biased than multiple imputation 34 . In future work, we plan to investigate the extent to which this assumption may have impacted our results in statistical methodological research (as we have done previously) 35 …”

“… 34 In future work, we plan to investigate the extent to which this assumption may have impacted our results in statistical methodological research (as we have done previously). 35 …”

Vaccine beliefs, adverse effects, and quality of life in patients with cancer undergoing routine COVID-19 vaccination

“…Testing was performed for association between dose 3 OCCS score and worst participant-reported toxicity at doses 1-2, adjusting for baseline OCCS score by using ANCOVA within each imputed dataset, and combining across imputations using the D2 statistic 21 . A flexible association between participant-reported toxicity incidence and selected QoL scores was allowed by fitting logit-link binomial generalized additive models to each imputed dataset, taking the median p-value from the approximate test of no association as an overall test of significance 22 . The association between participant-reported AEs and baseline QoL was assessed for adult patients only, due to small numbers of children.…”

Scanpro: robust proportion analysis for single cell resolution data

“…The beta coefficients of linear models from 100 simulations are pooled together using Rubin's rule [29], to give mean estimates of the proportions of each cluster. Moreover, the median values of the adjusted p-values are calculated as a final estimation, since it was shown that the median of multiple p-values from testing on multiple replicated datasets is a reliable estimate [30,31].…”