Measuring patients’ priorities using the Analytic Hierarchy Process in comparison with Best-Worst-Scaling and rating cards: methodological aspects and ranking tasks

Abstract

BackgroundIdentifying patient priorities and preference measurements have gained importance as patients claim a more active role in health care decision making. Due to the variety of existing methods, it is challenging to define an appropriate method for each decision problem. This study demonstrates the impact of the non-standardized Analytic Hierarchy Process (AHP) method on priorities, and compares it with Best-Worst-Scaling (BWS) and ranking card methods.MethodsWe investigated AHP results for different Consistency Ratio (CR) thresholds, aggregation methods, and sensitivity analyses. We also compared criteria rankings of AHP with BWS and ranking cards results by Kendall’s tau b.ResultsThe sample for our decision analysis consisted of 39 patients with rare diseases and mean age of 53.82 years. The mean weights of the two groups of CR ≤ 0.1 and CR ≤ 0.2 did not differ significantly. For the aggregation by individual priority (AIP) method, the CR was higher than for aggregation by individual judgment (AIJ). In contrast, the weights of AIJ were similar compared to AIP, but some criteria’s rankings differed. Weights aggregated by geometric mean, median, and mean showed deviating results and rank reversals. Sensitivity analyses showed instable rankings. Moderate to high correlations between the rankings resulting from AHP and BWS.LimitationsLimitations were the small sample size and the heterogeneity of the patients with different rare diseases.ConclusionIn the AHP method, the number of included patients is associated with the threshold of the CR and choice of the aggregation method, whereas both directions of influence could be demonstrated. Therefore, it is important to implement standards for the AHP method. The choice of method should depend on the trade-off between the burden for participants and possibilities for analyses.Electronic supplementary materialThe online version of this article (doi:10.1186/s13561-016-0130-6) contains supplementary material, which is available to authorized users.

Shaping an Effective Health Information Website on Rare Diseases Using a Group Decision-Making Tool: Inclusion of the Perspectives of Patients, Their Family Members, and Physicians

“…The results range is displayed in Figure 1 and shows the potential sensibility of local weights to outliers. The ranking results were calculated based on the geometric means because the literature suggests that this procedure is more precise [ 27 ]. However, the following box plots show the range of results in a more intuitive manner, displaying the average mean, as well as the maximum and minimum local weights.…”

“…AHP offers a direct approach, whereas conjoint analysis compares different attributes in combination, thereby leading to an indirect calculation of weights. Furthermore, it is more intuitive and easier to understand for inexperienced participants compared with other techniques (eg, the analytic network process [ 26 ] but more informative than other techniques, eg, best-worst scaling, ranking) [ 27 ]). Quantitative preference distances make extensive evaluation of preference structures possible [ 20 , 28 ].…”

“…Afterwards, individual opinions were summarized using an aggregation of individual priorities method. As literature suggests that values must correspond to reciprocal values of individual participants, weights were aggregated choosing the geometric means calculation [ 27 ]. As priority values need to sum up to one, resulting local priorities were weighted accordingly.…”

Analytic Hierarchy Process as a Tool to Explore the Success Factors of BIM Deployment in Construction Firms

“…value < 0.2). As in other engineering fields, Zhuang et al [8], Schmidt et al [9] and other scholars believe that an AHP questionnaire is acceptable when the C.R. value of a pairwise comparison matrix it infers is less than 0.2.…”

Tensions in sustainable warehousing: including the blue-collar perspective on automation and ergonomic workplace design

“…2 Generally, AHP studies already yield sensible results with few participants (cf., Whitaker 2007) while smaller samples are quite common in extant literature (e.g., Peterson et al 1994;Al-Harbi 2001;Pishchulov et al 2019). It is more decisive for the reliability of the AHP results that the alternatives are defined crisply as well as to assure the homogeneity of the participant group (Schmidt et al 2016). The present study assured a clear introduction to the process and a common understanding among the respondents.…”