Which interactions matter in economic evaluations? A systematic review and simulation study

Abstract

Background:We aimed to assess the magnitude of interactions in costs, quality-adjusted life-years (QALYs) and net benefits within a sample of published economic evaluations of factorial randomised controlled trials (RCTs), evaluate the impact that different analytical methods would have had on the results and compare the performance of different criteria for identifying which interactions should be taken into account. Methods:We conducted a systematic review of full economic evaluations conducted alongside factorial RCTs and reviewed the methods used in different studies, as well as the incidence, magnitude, statistical significance, and type of interactions observed within the trials. We developed the interaction-effect ratio as a measure of the magnitude of interactions relative to main effects. For those studies reporting sufficient data, we assessed whether changing the form of analysis to ignore or include interactions would have changed the conclusions. We evaluated how well different criteria for identifying which interactions should be taken into account in the analysis would perform in practice, using simulated data generated to match the summary statistics of the studies identified in the review.Results: Large interactions for economic endpoints occurred frequently within the 40 studies identified in the review, although interactions rarely changed the conclusions. Conclusions:Simulation work demonstrated that in analyses of factorial RCTs, taking account of all interactions or including interactions above a certain size (regardless of statistical significance) minimised the opportunity cost from adopting treatments that do not in fact have the highest true net benefit.

Economic Evaluation of Factorial Trials: Cost-Utility Analysis of the Atorvastatin in Factorial With Omega EE90 Risk Reduction in Diabetes 2 × 2 × 2 Factorial Trial of Atorvastatin, Omega-3 Fish Oil, and Action Planning

“…These interactions, however, did not change the conclusions at a £20 000/QALY ceiling ratio because atorvastatin is extremely cost-effective, whereas omega-3 is poor value for the money and action planning had little effect on either costs or QALYs. Although many trials are robust to the assumptions about interactions, 27 , 28 the conclusions of others are sensitive to which interactions are included. 29 , 30 The methods for dealing with the factorial design and the assumptions made about interactions are likely to have a larger effect in studies where all treatments affect QALYs and where cost-effectiveness ratios are close to the ceiling ratio.…”

“…Like 20% of published factorial trial-based economic evaluations, 27 AFORRD includes cluster randomization. In general, multilevel models or two-stage bootstrap procedures are the most robust methods for dealing with clustering.…”