Using the Analytic Hierarchy Process to Examine Judgment Consistency in a Complex Multiattribute Task

Abstract

This paper investigates the impact of framing and time pressure on human judgment performance in a complex multiattribute judgment task. We focus on the decision process of human participants who must choose between pairwise alternatives in a resource-allocation task. We used the Analytic Hierarchy Process (AHP) to calculate the relative weights of the four alternatives (i.e., C 1 , C 2 , C 3 , and C 4 ) and the judgment consistency. Using the AHP, we examined two sets of hypotheses that address the impact of task conditions on the weight prioritization of choice alternatives and the internal consistency of the judgment behavior under varying task conditions. The experiment simulated the allocation of robotic assets across the battlefield to collect data about an enemy. Participants had to make a judgment about which asset to allocate to a new area by taking into account three criteria related to the likelihood of success. We manipulated the information frame and the nature of the task. We found that, in general, participants gave significantly different weights to the same alternatives under different frames and task conditions. Specifically, in terms of ln-transformed priority weights, participants gave significantly lower weights to C 2 and C 4 and higher weight to C 3 under gain frame than under loss frame, and also, under different task conditions (i.e., Tasks #1, #2, and #3), participants gave significantly higher weight to C 4 in Task #1, lower weights to C 1 and C 4 , higher weight to C 3 in Task #2, and lower weight to C 3 in Task #3. Furthermore, we found that the internal consistency of the decision behavior was worse, first, in the loss frame than the gain frame and, second, under time pressure. Our methodology complements utility-theoretic frameworks by assessing judgment consistency without requiring the use of task-performance outcomes. This work is a step toward establishing a coherence criterion to investigate judgment under naturalistic conditions. The results will be useful for the design of multiattribute interfaces and decision aiding tools for real-time judgments in time-pressured task environments.

Integration of Distributed Generations in Smart Distribution Networks Using Multi-Criteria Based Sustainable Planning Approach

“…In addition to main objective of deploying DGs in ADN to reduce power loss P Loss , and enhance voltage profile V min , other PI such as Q Loss , C E Loss , C AIC , PE, PW, LU, LA, and SA are also considered in MCSP approach for TEES evaluation. The weights for each criterion are considered biased (unequal) and calculated by AHP [46]. Selected alternatives, attributes, the priority or relevance of each attribute, and performance ratings are all included in the decision matrix.…”

Aiding decision makers to reopening of places of worship

“…AHP is a general theory of measurement used to derive ratio scales from human judgments through multiple paired comparisons in a hierarchically structured task (Saaty, 1980). AHP has been used in human factors applications such as to rank‐order computer interfaces (Mitta, 1993), to ascertain appropriate knowledge elicitation methods (Chao et al, 1999), to select attributes for designing virtual environment systems (Stanney et al, 2003), and to analyze the decision process itself in multiattribute decisions (Jin et al, 2010; Spires, 1991).…”

A Node Selection Paradigm for Crowdsourcing Service Based on Region Feature in Crowd Sensing

“…It is a combination of qualitative and quantitative decision analysis methods. It is often used in multiobjective, multicriteria, multielement, multilevel unstructured complex decision-making problems, especially strategic decisionmaking issues, it has a wide range of practicality [14,15]. When the system releases a sensing task, the matching degrees of the task and the local feature vector of each region are calculated to obtain a certain number of services nodes with higher matching degrees.…”