When to Use Discrete Event Simulation (DES) for the Economic Evaluation of Health Technologies? A Review and Critique of the Costs and Benefits of DES

Abstract

Modelling in economic evaluation is an unavoidable fact of life. Cohort-based state transition models are most common, though discrete event simulation (DES) is increasingly being used to implement more complex model structures. The benefits of DES relate to the greater flexibility around the implementation and population of complex models, which may provide more accurate or valid estimates of the incremental costs and benefits of alternative health technologies. The costs of DES relate to the time and expertise required to implement and review complex models, when perhaps a simpler model would suffice. The costs are not borne solely by the analyst, but also by reviewers. In particular, modelled economic evaluations are often submitted to support reimbursement decisions for new technologies, for which detailed model reviews are generally undertaken on behalf of the funding body. This paper reports the results from a review of published DES-based economic evaluations. Factors underlying the use of DES were defined, and the characteristics of applied models were considered, to inform options for assessing the potential benefits of DES in relation to each factor. Four broad factors underlying the use of DES were identified: baseline heterogeneity, continuous disease markers, time varying event rates, and the influence of prior events on subsequent event rates. If relevant, individual-level data are available, representation of the four factors is likely to improve model validity, and it is possible to assess the importance of their representation in individual cases. A thorough model performance evaluation is required to overcome the costs of DES from the users' perspective, but few of the reviewed DES models reported such a process. More generally, further direct, empirical comparisons of complex models with simpler models would better inform the benefits of DES to implement more complex models, and the circumstances in which such benefits are most likely.

Identification, Review, and Use of Health State Utilities in Cost-Effectiveness Models: An ISPOR Good Practices for Outcomes Research Task Force Report

“…The number of discrete health states included in the model could lead to a decision to use discrete event simulation. 35 If a simpler model structure is used that does not represent all potentially important HSUs, the potential effects of such omissions should be examined and discussed.…”

Validation of a decision support model for investigation and improvement in stroke thrombolysis

“…The reasons for such a choice are three-fold: firstly, as in our study we focus on the validation of the model that is statistical/analytical in nature, where simulation is used mainly for conducting probabilistic sensitivity analyses, it is more relevant to review the literature focusing on validation of similar "non-simulation" models; secondly, the domain of simulation modeling within OR/MS is well known for its careful attention to detailed model validation ( Balci, 1989;Sargent, 2001Sargent, , 2013Whitner & Balci, 1989 ), often focusing on highly simulationspecific issues (e.g. visual walkthroughs, etc) that could be less relevant to "non-simulation" focused studies; and thirdly, as discussed by Brailsford and Vissers (2011 ), the domain of health care simulation modeling applications is expanding by the rate of up to 30 papers per day and conducting a comprehensive review of such a body of literature deserves its own special focus (such as in, e.g., ( Fone et al, 2003 ) and ( Karnon & Afzali, 2014 )) and would have been impossible within the scope of this study.…”

Discrete Event Simulation-Based Resource Modelling in Health Technology Assessment

“…The use of DES is also gaining momentum within the field of HTA itself [12][13][14]. There has been a recent systematic review on the use of DES for HTA [15], which identified 42 relevant studies. However, they excluded the studies that modelled capacity constraints.…”

“…Other than the few studies that were identified in our review [19,20,26,28], most HTA studies do not address the effects of constraints explicitly when developing a patient-level model [15]. The reviewed studies showed that DES is an effective tool to assess the effects of constraints, but given the small number of studies found in our review, there is a gap in understanding on how DES can be used for considering explicit constraints when performing RM in HTA.…”