Comparing institutional-level bibliometric research performance indicator values based on different affiliation disambiguation systems

Abstract

The present study is an evaluation of three frequently used institution name disambiguation systems. The Web of Science normalized institution names and Organization Enhanced system and the Scopus Affiliation ID system are tested against a complete, independent institution disambiguation system for a sample of German public sector research organizations. The independent system is used as the gold standard in the evaluations that we perform. We study the coverage of the disambiguation systems and, in particular, the differences in a number of commonly used bibliometric indicators. The key finding is that for the sample institutions, the studied systems provide bibliometric indicator values that have only a limited accuracy. Our conclusion is that for any use with policy implications, additional data cleaning for disambiguating affiliation data is recommended.

Open access uptake in Germany 2010–2018: adoption in a diverse research landscape

“…The Web of Science in-house database maintained by the German Competence Center for Bibliometrics (WoS-KB), 2019 version, was used to determine the publication output of German universities and non-university research institutions. The main advantage of the WoS-KB for the purpose of our study is its disambiguated address information (Donner et al, 2020;Rimmert et al, 2017), which not only allowed obtaining the publication output at the institutional level, but also to interlink them to the specific sectors of the German research system. The institutional disambiguation authority file was developed at the Institute for Interdisciplinary Studies of Science at Bielefeld University and works as a central component with a "near-complete national-scale coverage" of Germany's institutions represented in the Web of Science (Donner et al, 2020).…”

“…The institutional disambiguation authority file was developed at the Institute for Interdisciplinary Studies of Science at Bielefeld University and works as a central component with a "near-complete national-scale coverage" of Germany's institutions represented in the Web of Science (Donner et al, 2020). Accordingly, Donner et al (2020) reported a very high accuracy. The disambiguated affiliation system for German institutions can, thus, serve as a gold standard for institution name disambiguation.…”

A quantitative view of the structure of institutional scientific collaborations using the example of Berlin

“…This affects the results of macro studies depending on the database employed, subset of scientific publications used, document types analyzed, and level of aggregation and normalization applied (Stahlschmidt et al, 2019;Stephen et al, 2020). In terms of cleanness of the data, there is a strong need for disambiguation of scientific entity names (e.g., authors, organizations) which might bias the quality of results (Aman, 2018;D'Angelo & van Eck, 2020;Donner, Rimmert, & van Eck, 2019). Thus, one of our methodological goals is to introduce organization name disambiguation techniques to match scientific organization names with publicly available databases (e.g., Wikidata and Global Research Identifier Database (GRID)) and evaluate the reliability of the results in comparison to established techniques.…”

“…The data delivered by Scopus is not perfect. It is prone to error and there is a strong need for disambiguation of organization names (Donner et al, 2019). Without disambiguation, the coauthorship networks constructed will have multiple representations of the same actor and an artificially higher level of (dis)connectivity.…”

Machine Learning Technologies for Sustainability in Smart Cities in the Post-COVID Era

“…The result of this analytical process shows a complete map of the scientific production related to the search terms where, first of all, the resulting network of keywords can be seen (Figure 4). The following analyses were carried out [40,44]:…”