Multi-criteria decision analysis for health technology assessment: addressing methodological challenges to improve the state of the art

Abstract

Background Multi-criteria decision analysis (MCDA) concepts, models and tools have been used increasingly in health technology assessment (HTA), with several studies pointing out practical and theoretical issues related to its use. This study provides a critical review of published studies on MCDA in the context of HTA by assessing their methodological quality and summarising methodological challenges. Methods A systematic review was conducted to identify studies discussing, developing or reviewing the use of MCDA in HTA using aggregation approaches. Studies were classified according to publication time and type, country of study, technology type and study type. The PROACTIVE-S approach was constructed and used to analyse methodological quality. Challenges and limitations reported in eligible studies were collected and summarised; this was followed by a critical discussion on research requirements to address the identified challenges. Results 129 journal articles were eligible for review, 56% of which were published in 2015-2017; 42% focused on pharmaceuticals; 36, 26 and 18% reported model applications, issues regarding MCDA implementation analyses, and proposing frameworks, respectively. Poor compliance with good methodological practice (< 25% complying studies) was found regarding behavioural analyses, discussion of model assumptions and uncertainties, modelling of value functions, and dealing with judgment inconsistencies. The five most reported challenges related to evidence and data synthesis; value system differences and participant selection issues; participant difficulties; methodological complexity and resource balance; and criteria and attributes modelling. A critical discussion on ways to address these challenges ensues. Discussion Results highlight the need for advancement in robust methodologies, procedures and tools to improve methodological quality of MCDA in HTA studies. Research pathways include developing new model features, good practice guidelines, technologies to enable participation and behavioural research. Keywords Multi-criteria decision analysis • Health technology assessment • Systematic review • Methodological quality • Methodological challenges • MCDA modelling JEL Classification I (Health education and welfare) • I10 (General) • I18 (Government policy regulation public health) • C44 (Operations research statistical decision theory) • D80 (General)

Assessing the Preferences for Criteria in Multi-Criteria Decision Analysis in Treatments for Rare Diseases

“…MCDA frameworks are complex models with increasing complexity as more criteria are added to the decision-making process and additional evidence generation is required (71). Some people might consider that replacing a cost-effectiveness analysis with MCDA results in added uncertainty of what the results really imply in the context of reimbursement decisions.…”

The Value of Different Types of Economic and Budget Analysis for Informing Real World Decision Making: the Case of Cervical Cancer Screening in South Africa

“…Considering this, the scales used to score questions in the MCDA most certainly had an impact on the nal result. This difference in value systems between experts and stakeholders has previously been noted as an issue which needs to be addressed (28).…”

Comprehensive value assessment of drugs using a multi-criteria decision analysis: An example of targeted therapies for metastatic colorectal cancer treatment

“…Drug value evaluations should consider multiple dimensions and criteria. [50, 51] The NHI formulary listing of drugs often takes into consideration the comparative efficacy, comparative safety, cost-effectiveness, and overall expenditures. However, the results showed that if only the four criteria above are considered, only 57.5% of the weights are covered.…”