Some heuristics for the consensus ranking problem

Beck, Michael P.; Lin, Benjamin W.

doi:10.1016/0305-0548(83)90021-7

Cited by 44 publications

(36 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the KSC shuttle project engineering office wanted the final ranking of the projects to be closest to the DMs' preferences and also to yield the greatest number of agreements. Beck and Lin's (1983) definition of agreement meets these requirements. If a DM ranks project m above project n and project m is also ranked above project n in the final consensus ranking, this counts as one agreement.…”

Section: Integration Phasementioning

confidence: 95%

See 1 more Smart Citation

CROSS: A Multicriteria Group-Decision-Making Model for Evaluating and Prioritizing Advanced-Technology Projects at NASA

Tavana

2003

Interfaces

View full text Add to dashboard Cite

Evaluating and prioritizing advanced-technology projects is a particularly difficult task for the staff at the Kennedy Space Center (KSC) shuttle project engineering office. Because the evaluation process is complex and unstructured, decision makers (DMs) must consider vast amounts of diverse information concerning safety, systems engineering, cost savings, process enhancement, reliability, and implementation. Intuitive methods developed in the past have helped them to use large volumes of information in evaluating projects. However, these intuitive methods do not provide a structured framework for systematic evaluation. CROSS (consensus-ranking organizational-support system) is a multicriteria group-decision-making model that I implemented successfully at KSC to capture the DMs' beliefs through sequential, rational, and analytical processes. CROSS uses the analytic hierarchy process (AHP), subjective probabilities, the entropy concept, and the maximize-agreement heuristic (MAH) to enhance the DMs' intuition in evaluating sets of projects. (Government: programs. Decision analysis: multiple criteria.) T he rapid development of technology over the last few decades and the increased awareness of its effects on society have focused critical attention on government agencies that support technology development. The public is concerned with the governance of these agencies and with obtaining the maximum return from public investment in advanced technology. Public pressure has forced Congress to mandate the National Aeronautic and Space Administration (NASA) to be more accountable in its evaluation of advanced-technology projects. The demand for accountability, the pressure to cut costs, and the increasing number of projects have made evaluating projects extremely difficult.Over the last several decades, analysts have developed a philosophy and a body of intuitive and analytical models to help decision makers (DMs) to evaluate and select projects. However, the intuitive models do not offer a structured framework for evaluating projects systematically, while the analytical models are not intended to capture intuitive preferences. The literature on project selection contains hundreds of models, including scoring methods, economic methods, portfolio methods, and decision analysis.Scoring methods use algebraic formulas to produce a score for each project (Lockett et al. 1984, Melachrinoudis and Rice 1991, Moore and Baker 1969. Economic methods use financial models to calculate the monetary payoff of each project under consideration (Graves and Ringuest 1991, Mehrez 1988). Portfolio methods evaluate the entire set

show abstract

Section: Integration Phasementioning

confidence: 95%

“…In CROSS, I used Microsoft Excel and MAH to provide a consensus ranking of the projects. MAH, proposed by Beck and Lin (1983), forms consensus orderings that reflect collective DM agreement given ordinal or cardinal project rankings. Similarly, one could use normative averaging procedures to combine individual rankings.…”

Section: Integration Phasementioning

confidence: 99%

CROSS: A Multicriteria Group-Decision-Making Model for Evaluating and Prioritizing Advanced-Technology Projects at NASA

Tavana

2003

Interfaces

View full text Add to dashboard Cite

show abstract

“…The similarity of consensus rankings generated by the different algorithms is largely unknown. We examine the similarity in rankings generated by our method with the best-known method of BAK and two other commonly used techniques proposed by Beck and Lin [33] and Cook and Kress [22]. We use a Monte Carlo simulation to examine the extent to which these algorithms yield similar rank ordering across a range of problems with k DMs and n alternatives.…”

Section: Introductionmentioning

confidence: 99%

“…Beck and Lin [33] have developed a procedure for approximating the optimal consensus rankings known as maximize agreement heuristic (MAH). MAH is commonly used in practice because of its simplicity, flexibility, and general performance (see Kengpol and Tuominen [34], Lewis and Butler [35], Tavana et al [36], Tavana [37,38]).…”

Section: Introductionmentioning

confidence: 99%

“…The proposed hybrid model combines parts of the two commonly used consensus ranking techniques of Beck and Lin (1983) and Cook and Kress (1985) into an intuitive and computationally simple model. We illustrate our method and then run a Monte Carlo simulation across a range of k and n to compare the similarity of the consensus rankings generated by our method with the best-known method of Borda and Kendall (Kendall 1962) and the two methods proposed by Beck and Lin (1983) and Cook and Kress (1985). DCM and Beck and Lin's method yielded the most similar consensus rankings, whereas the Cook-Kress method and the Borda-Kendall method yielded the least similar consensus rankings.…”

mentioning

confidence: 99%

See 1 more Smart Citation

A Hybrid Distance-Based Ideal-Seeking Consensus Ranking Model

Tavana

LoPinto

Smither

2007

Journal of Applied Mathematics and Decision Sciences

View full text Add to dashboard Cite

Ordinal consensus ranking problems have received much attention in the management science literature. A problem arises in situations where a group of k decision makers (DMs) is asked to rank order n alternatives. The question is how to combine the DM rankings into one consensus ranking. Several different approaches have been suggested to aggregate DM responses into a compromise or consensus ranking; however, the similarity of consensus rankings generated by the different algorithms is largely unknown. In this paper, we propose a new hybrid distance-based ideal-seeking consensus ranking model (DCM). The proposed hybrid model combines parts of the two commonly used consensus ranking techniques of Beck and Lin (1983) and Cook and Kress (1985) into an intuitive and computationally simple model. We illustrate our method and then run a Monte Carlo simulation across a range of k and n to compare the similarity of the consensus rankings generated by our method with the best-known method of Borda and Kendall (Kendall 1962) and the two methods proposed by Beck and Lin (1983) and Cook and Kress (1985). DCM and Beck and Lin's method yielded the most similar consensus rankings, whereas the Cook-Kress method and the Borda-Kendall method yielded the least similar consensus rankings.

show abstract

A new rank correlation coefficient with application to the consensus ranking problem

Emond

Mason

2002

Multi Criteria Decision Anal

137

140

View full text Add to dashboard Cite

The consensus ranking problem has received much attention in the statistical literature. Given m rankings of n objects the objective is to determine a consensus ranking. The input rankings may contain ties, be incomplete, and may be weighted. Two solution concepts are discussed, the first maximizing the average weighted rank correlation of the solution ranking with the input rankings and the second minimizing the average weighted Kemeny-Snell distance. A new rank correlation coefficient called t x is presented which is shown to be the unique rank correlation coefficient which is equivalent to the Kemeny-Snell distance metric. The new rank correlation coefficient is closely related to Kendall's tau but differs from it in the way ties are handled. It will be demonstrated that Kendall's t b is flawed as a measure of agreement between weak orderings and should no longer be used as a rank correlation coefficient. The use of t x in the consensus ranking problem provides a more mathematically tractable solution than the Kemeny-Snell distance metric because all the ranking information can be summarized in a single matrix. The methods described in this paper allow analysts to accommodate the fully general consensus ranking problem with weights, ties, and partial inputs.

show abstract

Some heuristics for the consensus ranking problem

Cited by 44 publications

References 7 publications

CROSS: A Multicriteria Group-Decision-Making Model for Evaluating and Prioritizing Advanced-Technology Projects at NASA

CROSS: A Multicriteria Group-Decision-Making Model for Evaluating and Prioritizing Advanced-Technology Projects at NASA

A Hybrid Distance-Based Ideal-Seeking Consensus Ranking Model

A new rank correlation coefficient with application to the consensus ranking problem

Contact Info

Product

Resources

About