Uncertainty and Coverage With Evidence Development: Does Practice Meet Theory?

Abstract

In theory, a successful coverage with evidence development (CED) scheme is one that addresses the most important uncertainties in a given assessment. We investigated the following: (1) which uncertainties were present during the initial assessment of 3 Dutch CED cases, (2) how these uncertainties were integrated in the initial assessments, (3) whether CED research plans included the identified uncertainties, and (4) issues with managing uncertainty in CED research and ways forward from these issues.Methods: Three CED initial assessment dossiers were analyzed and 16 stakeholders were interviewed. Uncertainties were identified in interviews and dossiers and were categorized in different causes: unavailability, indirectness, and imprecision of evidence. Identified uncertainties could be mentioned, described, and explored. Issues and ways forward to address uncertainty in CED schemes were discussed during the interviews.Results: Forty-two uncertainties were identified. Thirteen (31%) were caused by unavailability, 17 (40%) by indirectness, and 12 (29%) by imprecision. Thirty-four uncertainties (81%) were only mentioned, 19 (45%) were described, and the impact of 3 (7%) uncertainties on the results was explored in the assessment dossiers. Seventeen uncertainties (40%) were included in the CED research plans. According to stakeholders, research did not address the identified uncertainty, but CED research should be designed to focus on these. Conclusions:In practice, uncertainties were neither systematically nor completely identified in the analyzed CED schemes. A framework would help to systematically identify uncertainty, and this process should involve all stakeholders. Value of information analysis, and the uncertainties that are not included in this analysis should inform CED research design.

How to Value Orphan Drugs? A Review of European Value Assessment Frameworks

“…However, a recent OECD report found that, to date, most performance-based MEAs have failed to decrease clinical uncertainty (Morel et al, 2013;Wenzl and Chapman 2019). This results, among other reasons, from the fact that CED protocols generally fail to address the majority of the uncertainties which were initially identified (Pouwels et al, 2019). The authors further suggest that jurisdictions implement legislation that (at least to some extent) allows them to share information on effectiveness between countries, an idea that seems particularly relevant to rare diseases.…”

Value of Information Analysis: Are We There Yet?

“…They may also be sceptical about the robustness of the findings of VOI with the associated modelling and assumptions. Nevertheless, incorporating this approach into decision frameworks is intuitive if it is introduced as a decision support tool to reduce the chance and costly consequences of sub-optimal decisions [ 1 , 2 , 29 ]. Many decision-making organisations embrace evidence-based approaches in making decisions using cost-effectiveness analysis; furthermore, handling and presenting uncertainty is already embedded in the current decision frameworks in many jurisdictions.…”

Exploratory, Participatory and Iterative Assessment of Value: A Response to Recent Commentaries

“…3 In the Netherlands, conditional reimbursement schemes with evidence collection are hardly based on uncertainties other than imprecision. 12 With use of exploratory modeling, one could explore under which circumstances (eg, at which survival rate, or in case of which parallel developments in the control of the relevant disease condition) ipilimumab is cost-effective. An adaptive strategy can then be implemented where evidence is collected and clear thresholds are set: if a certain threshold (eg, survival rate) is reached, reimbursement of the drug can be continued, if not, reimbursement should be withdrawn.…”