Experiences of Structured Elicitation for Model-Based Cost-Effectiveness Analyses

Abstract

BackgroundEmpirical evidence supporting the cost-effectiveness estimates of particular health care technologies may be limited, or it may even be missing entirely. In these situations, additional information, often in the form of expert judgments, is needed to reach a decision. There are formal methods to quantify experts’ beliefs, termed as structured expert elicitation (SEE), but only limited research is available in support of methodological choices. Perhaps as a consequence, the use of SEE in the context of cost-effectiveness modelling is limited.ObjectivesThis article reviews applications of SEE in cost-effectiveness modelling with the aim of summarizing the basis for methodological choices made in each application and recording the difficulties and challenges reported by the authors in the design, conduct, and analyses.MethodsThe methods used in each application were extracted along with the criteria used to support methodological and practical choices and any issues or challenges discussed in the text. Issues and challenges were extracted using an open field, and then categorised and grouped for reporting.ResultsThe review demonstrates considerable heterogeneity in methods used, and authors acknowledge great methodological uncertainty in justifying their choices. Specificities of the context area emerging as potentially important in determining further methodological research in elicitation are between- expert variation and its interpretation, the fact that substantive experts in the area may not be trained in quantitative subjects, that judgments are often needed on various parameter types, the need for some form of assessment of validity, and the need for more integration with behavioural research to devise relevant debiasing strategies.ConclusionsThis review of experiences of SEE highlights a number of specificities/constraints that can shape the development of guidance and target future research efforts in this area.

Estimating the Marginal Productivity of the English National Health Service From 2003 to 2012

“…Clearly, linking the estimated effects on mortality to QALYs requires a number of assumptions to be made. For a more detailed summary of all assumptions, their justification, and a discussion of their likely impact on the central estimate of the cost per QALY, see Table 32 in Claxton et al 5(p 83) Recently, the plausibility of these assumptions has been examined through structured elicitation from clinical experts, 14 and this work suggests that these assumptions are likely to be conservative with respect to the QALY effects of changes in expenditure (ie, the cost per QALY is likely to be lower than that estimated using these assumptions).…”

Health Opportunity Costs: Assessing the Implications of Uncertainty Using Elicitation Methods with Experts

“…We demonstrate the applicability of the elicitation exercise in practice. Its design draws from wider experience of elicitation in health technology assessment 14 and literature from other areas of science (for example, refs. 15 and 16).…”

Health economic evaluation of gene replacement therapies: methodological issues and recommendations

“…Several elicitation procedures are available to obtain information from experts and make a probabilistic representation of their knowledge. Two different approaches of elicitation are commonly used in the literature of structured elicitations for cost-effectiveness analyses: 1) the fixed interval method, in which the expert reports his/her probability of the uncertain quantity of interest θ, for example the recurrence rate, the mortality rate or the time to death, lying in specified intervals, and 2) the variable interval method, in which he/she makes quantile judgements[ 80 ]. Among the common fixed interval elicitation methods, is the trial roulette method, also called the ‘chips and bins method’, where the expert provides probabilities of θ lying in a particular ‘bin’ by allocating ‘chips’ to that bin [ 81 , 82 ].…”

“… Two different approaches of structured elicitation are recommended for cost-effectiveness analyses: 1) the fixed interval method, in which the expert reports his/her probability of the uncertain quantity of interest θ, for example the recurrence rate, the mortality rate or the time to death, lying in specified intervals, and 2) the variable interval method, in which (s)he makes quantile judgements. Panel**, Grigore 2016[ 79 ], Soares 2018[ 80 ] → Extrapolation methods Standard modelling techniques used in economic evaluation, such as Markov models and Discrete Event Simulation, will likely be appropriate for GRT. The challenge will be to find appropriate data, such as transition probabilities, to populate these models.…”