Measurability invariance, continuity and a portfolio representation

On the internal dynamics of the Shanghai ranking

“…If a measure is required, our findings support the use of the ARWU indicators as one set of measures of this quality, and of the ARWU ranking as a summary thereof. As discussed by Sawyer et al (2013) the process of measurement in the social sciences, when construct validity cannot be assessed by benchmarking, necessarily includes social processes of persuasion and convergence alongside operational processes of estimation. An objective of social science measurement is to ensure that others acknowledge the measures as benchmarks replacing the unknown and unknowable measure.…”

“…Ranking and clustering algorithms used in rank-driven sequencing are designed to harmonise disparate measures and distributions of marks before aggregation by converting them to quasi-uniform distributions that retain no information about the original measured values beyond their sequence. Although the measures are originally quantitative and disparate, a rankdriven sequence becomes agnostic about the actual measured values (Sawyer et al, 2013).…”

“…ARWU can be properly classified among the score-driven tables, but might it perhaps be preferable to employ a rank-driven procedure? The choice evidently turns on the meaningfulness of the actual indicator values as measurements of performance (Sawyer et al, 2013), and on their suitability for aggregation. A scoredriven procedure is indicated if the indicator measurements have meaning and can be combined, although the procedure is contingent of finding conversion strategies to equalize between indicators.…”

“…University ranking on an international scale is a relatively recent phenomenon, and methodological problems are not unexpected (Sawyer et al, 2013). However, discussions around ARWU and other ranking systems tend to be either so highly critical that any potential usefulness is dismissed, or so willingly accepting that obvious flaws are overlooked.…”

On the effects of institutional size in university classifications: the case of the Shanghai ranking

“…The ARWU raises controversy about both kinds of issue, for example the validity of the concept of ranking universities by a single number is hotly disputed, while the issue of measuring institutions defined through a complex concordance of names remains problematic. Sawyer et al (2013) describe measurements like the ARWU ranking as a tournament involving ''convergence towards a proposed measure'' through minimization of measurement error, the persuasion of others, convergence of other measures to the proposed measure and anchoring of the proposed measure. In relation to university ranking, the tournament features activities such as size/ excellence debates (Zitt and Filliatreau 2007), defensive moves by bibliometric experts (van Raan 2005), the exercise of public political influence (Boulton 2010) and conflicts between advocates of unitary versus pluralist presentation of results (van Vught and Ziegele 2011).…”