Aggregating multiple ordinal rankings in engineering design: the best model according to the Kendall’s coefficient of concordance

Abstract

Aggregating the preferences of a group of experts is a recurring problem in several fields, including engineering design; in a nutshell, each expert formulates an ordinal ranking of a set of alternatives and the resulting rankings should be aggregated into a collective one. Many aggregation models have been proposed in the literature, showing strengths and weaknesses, in line with the implications of Arrow's impossibility theorem. Furthermore, the coherence of the collective ranking with respect to the expert rankings may change depending on: (i) the expert rankings themselves and (ii) the aggregation model adopted. This paper assesses this coherence for a variety of aggregation models, through a recent test based on the Kendall's coefficient of concordance (W), and studies the characteristics of those models that are most likely to achieve higher coherence. Interestingly, the so-called Borda count model often provides best coherence, with some exceptions in the case of collective rankings with ties. The description is supported by practical examples.

Advantages and Disadvantages of the Remote Teamwork

“…Kendall's coefficient of concordance). Šiuo metodu yra įvertinamas ekspertų atsakymų suderinamumas ir išsiaiškinama, ar nėra išskirčių, kurias reikėtų pašalinti (Franceschini & Maisano, 2021). Duomenis vertinant Kendall konkordancijos koeficientu pirmiausia išsikeliamos hipotezės (Ahudey et al, 2020):…”

A proposal for an operational methodology to assist the ranking-aggregation problem in manufacturing

“…(i) the m-expert rankings, and (ii) the collective ranking obtained after the application of a given aggregation model (k) to the previous expert rankings. Consistency between collective ranking and expert rankings is assessed in relative terms, by comparing 𝑊 𝑘 (𝑚+1) with the traditional W. 𝑊 𝑘 (𝑚+1) ≥ 𝑊 denotes consistency (or positive consistency) between the collective ranking and the m-rankings, while 𝑊 𝑘 (𝑚+1) < 𝑊 denotes inconsistency (or negative consistency) (Franceschini and Maisano, 2021).…”