Making Scientometric and Econometric Sense out of Nirf 2017 Data

Abstract

We perform a comparative end-to-end research evaluation of leading engineering institutions in India separating out the bibliometric part of the chain from the econometric part. This combines size-dependent and size-independent terms based on quantity and quality (impact) in a meaningful way. Output or outcome at the bibliometric level is measured using a second-order composite indicator, and the productivity or efficiency terms follow accordingly using the input to output or outcome factors. Data are taken from the recent release of the National Institutional Ranking Framework (NIRF) 2017 in the public domain. Thus, the ranking based on NIRF scores of the Indian Institute of Technology, Madras as the best engineering institution in India is too simplistic a conclusion.Keywords: Bibliometrics, comparative research evaluation, institutional ranking, size dependence and independence.THE National Institutional Ranking Framework (NIRF) has just released its 2017 rankings of higher educational institutions across the country. A wealth of scientometric and institutional data are now available in the public domain. In this communication, we focus only on the aspect of research excellence as measured by publications, citations and impact from three different bibliometric databases for the top 25 engineering institutions ranked in 2017. An end-to-end comparative research evaluation of leading engineering institutions in India is performed by separating out the bibliometric part (inner core) of the chain from the econometric part (outer shell). This combines size-dependent and size-independent terms based on quantity and quality (impact) in a meaningful way. Output or outcome at the bibliometric level is measured using a second-order composite indicator, and the productivity or efficiency terms follow accordingly using the input to output or outcome factors.Savithri and Prathap 1 showed that the research performance of leading higher education institutions in India and China can be summarized from the input end to the outcome end using six primary and secondary bibliometric indicators representing the entire chain of activity: input-output-excellence-outcome-productivity. Principal component analysis indicated that the primary indicators are orthogonal and represent size-dependent quantity and size-independent quality/productivity dimensions respectively. Composite indicators which combine size-dependent and size-independent terms were also needed to measure output and outcome. Data from the 2014 release of the SCImago Institutions Rankings (SIR) were used.Abramo and D'Angelo 2,3 have recently argued that the use of size-independent citation indicators from the bibliometric part (inner core) of the chain to rank institutions for performance must be combined with the productivity and efficiency measures from the econometric outer loop of assessment. This requires the bibliometric core of the chain (measuring output or outcome using bibliometric indicators) to be separated from the econometric part (the outcome or output to...