On using the Shanghai ranking to assess the research performance of university systems

Abstract

We take a new look at the Shanghai Jiao Tong Academic Ranking of World Universities to evaluate the performance of whole university systems. We deal with system aggregates by means of averaging scores taken over a number of institutions from each higher education system according to the Gross Domestic Product of its country. We treat the set of indicators (measures) at the country level as a scale, and investigate its reliability and dimensionality using appropriate statistical tools. After a Principal Component Analysis is performed, a clear picture emerges: at the aggregate level ARWU seems to be a very reliable one-dimensional scale, with a first component that explains more than 72% of the variance of the sample under analysis. The percentages of variance of the indicators explained by the first component do shed light on the fact that ARWU is in fact measuring the research quality (both at the individual and collective levels) of a university system. When the second principal component is taken into account, the two principal components contribute to explain more than 90% of the variance. The rotated solution facilitates the interpretation of the components and provides clear and interesting clustering information about the 32 higher education systems under analysis.

A study of global and local visibility as web indicators of research production

“…The position of universities in international rankings is today considered as a strategic issue for most universities. Although they begin as a tool to rank universities by comparing their research capacity and output, they are increasingly being used as a stick to measure institutions and not just in relation with research (Docampo 2011). However, there is a controversy regarding their methodologies and their choice of indicators.…”

Super-efficiency of education institutions: an application to economics departments

“…This method has been used in the literature before to classify determinants of research productivity. See for instance Ramesh Babu and Singh (1998), Costas and Bordons (2007), Franceschet (2009), Docampo (2011), or Ortega, Lopez-Romero, and Fernandez (2011. Cordero-Ferrera, Pedraja-Chaparro, and Santín-González (2010) use PCA in a DEA framework to condense information.…”

Review on the Impact of World Higher Education Rankings: Institutional Competitive Competence and Institutional Competence

“…Hence, they concluded that criteria were random and depended on availability. the quantity and quality of its scientific production and the excellence of its students and alumni, the information it provides, when properly used, allows us to gain a useful insight into the research performance of whole university systems (Docampo, 2010 (Merisotis, 2002). Salmi and Saroyan (2007) examined league tables with their similarities and their potential to be used as information for policymakers and students.…”