Consistency and Accuracy in Decision Aids: Experiments with Four Multiattribute Systems*

Abstract

There have been a number of multiattribute decision aids developed to aid selection problems. Multiattribute value theory and the analytic hierarchy process ~IE two commonly used techniques. Different systems can result in radically different conclusions if they inaccurately and inconsistently reflect the preference structure of decision makers, or if they are based on inappropriate theoretical models. This study examines the impact of the underlying theoretical model, the method in which preference information is elicited, and the structure of alternatives as influences on the results from using various decision aids. It was found that two systems based on the multiattribute value theory model were just as diverse in their conclusions as were results between AHP and the multiattribute value theory models. Therefore, accuracy of infomation reflecting decision maker preference is an important consideration. Feedback capable of assuring the decision maker that information provided is consistent is a necessary feature required of decision aids applied to selection problems. The study also found that the way in which information is elicited influenced the result more than did the underlying model. Exact numerical data for complex concepts such as attribute importance and alternative performance on attributes is not necessary, and elicitation procedures that axe more natural for the user are likely to be more accurate.

A multi‐criterion experimental comparison of three multi‐attribute weight measurement methods

“…Unfortunately, most studies have found low convergent validity between methods (e.g. Hobbs, 1980;Schoemaker and Waid, 1982;Adelman et al, 1984;Belton, 1986;Olson, 1995). However, with the possibility of determining theoretically valid weights using functional measurement methodology, it is now possible to compare the theoretical validity of the methods.…”

Tradeoffs in Design-to-Budget Projects

“…In the spirit of the MAUT method 1,2,3 , the problem was reformulated as a design and optimization problem: find out which products shall be offered so that the quality of the service is maximized under the constraint that the overall cost remains under a given limit. This is first a design problem in the sense that one searches for a combination of the products to offer.…”

Influence of models and scales on the ranking of multiattribute alternatives

“…It is reasonable to expect that alternatives with "reversed" preferences in the previous two models would be left incomparable in ZAPROS. Many studies show that stable comparisons of alternatives are provided by different methods only when alternatives are substantially different in value, but it is not the case when alternatives are close in value and should be considered incomparable (or equal) with the used criterion system Olson et al, 1995).…”