A research project ranking method based on independent reviews by using the principle of the distance to the perfectly assessed project

Abstract

Abstract. The paper discusses the problem of ranking research projects based on the assessment obtained from n ≥ 1 independent blinded reviewers. Each reviewer assesses several project features, and the total score is defined as the weighted arithmetic mean, where the weights of features are determined according to the well-known AHP method. In this way, it is possible to identify each project by a point in n-dimensional space. The ranking is performed on the basis of the distance of each project to the perfectly assessed project. Thereby the application of different metric functions is analyzed. We believe it is inappropriate to use a larger number of decimal places if two projects are almost equidistant (according to some distance function) to the perfectly assessed project. In that case, it would be more appropriate to give priority to the project that has received more uniform ratings. This can be achieved by combining different distance functions. The method is illustrated by several simple examples and applied by ranking internal research projects at Josip Juraj Strossmayer University of Osijek, Croatia.

CEJOR special issue of Croatian Operational Research Society

“…The special issue contains papers with wide range of relevant and contemporary topics in the area of OR which were also discussed by other authors in previous issues of this journal and other OR journals, such as the topics of IPM (Meszaros 2015), multicriteria optimization (Trdin and Bohanec 2018), social network analysis (Csató 2017), game theory (Aust and Buscher 2016;McCormick and Owen 2018). DEA (Šegota et al 2017), clustering (Scitovski et al 2017), and graph theory (Vinkó and Gelle 2017).…”