Propensity score prediction for electronic healthcare databases using super learner and high-dimensional propensity score methods

Abstract

The optimal learner for prediction modeling varies depending on the underlying data-generating distribution. Super Learner (SL) is a generic ensemble learning algorithm that uses cross-validation to select among a "library" of candidate prediction models. The SL is not restricted to a single prediction model, but uses the strengths of a variety of learning algorithms to adapt to different databases. While the SL has been shown to perform well in a number of settings, it has not been thoroughly evaluated in large electronic healthcare databases that are common in pharmacoepidemiology and comparative effectiveness research. In this study, we applied and evaluated the performance of the SL in its ability to predict treatment assignment using three electronic healthcare databases. We considered a library of algorithms that consisted of both nonparametric and parametric models. We also considered a novel strategy for prediction modeling that combines the SL with the high-dimensional propensity score (hdPS) variable selection algorithm. Predictive performance was assessed using three metrics: the negative log-likelihood, area under the curve (AUC), and time complexity. Results showed that the best individual algorithm, in terms of predictive performance, varied across datasets. The SL was able to adapt to the given dataset and optimize predictive performance relative to any individual learner. Combining the SL with the hdPS was the most consistent prediction method and may be promising for PS estimation and prediction modeling in electronic healthcare databases.

Should a propensity score model be super? The utility of ensemble procedures for causal adjustment

“…Super Learner has also been considered in the context of longitudinal data, where it has been found useful in the presence of model misspecification . Overall, Super Learner is now being widely adopted in the causal inference literature and in applications …”

“…14 Overall, Super Learner is now being widely adopted in the causal inference literature and in applications. [15][16][17][18] In much of this literature, and certainly in most applications, adjustment via the propensity score is achieved in a singly robust fashion, that is, designed to give consistent inference on the causal estimand under an assumption of correct specification (at least within a class of parametric, flexible, or ensemble procedures). Doubly robust procedures that provide consistent estimation if either the propensity score or a proposed conditional outcome mean model-as would be utilized in standard regression-are correctly specified are also well established in the statistical literature, but such procedures are not as widely adopted in practice.…”

Estimating inverse probability weights using super learner when weight-model specification is unknown in a marginal structural Cox model context

“…This is especially true when computationally intensive learners, such as bagged CART or boosted CART , are included in the candidate library . In general, the computation time for SL is at least twice the sum of all the candidate learners' computation time, considering fitting on the training sets, computing the corresponding weights from the validation sets, and fitting the entire data eventually . Similar to other MSCM simulation studies , we computed robust sandwich standard error in this paper.…”

A Super-Learner Ensemble of Deep Networks for Vehicle-Type Classification

“…Therefore, the weights of the super learner are calculated by minimizing the single-split cross-validated loss as suggested in [9]. Ju et al [34] show the success of the single-split super learner on three large healthcare databases.…”