A new methodology for comparing Google Scholar and Scopus

Abstract

A new methodology is proposed for comparing Google Scholar (GS) with other citation indexes. It focuses on the coverage and citation impact of sources, indexing speed, and data quality, including the effect of duplicate citation counts. The method compares GS with Elsevier's Scopus, and is applied to a limited set of articles published in 12 journals from six subject fields, so that its findings cannot be generalized to all journals or fields. The study is exploratory, and hypothesis generating rather than hypothesis-testing. It confirms findings on source coverage and citation impact obtained in earlier studies. The ratio of GS over Scopus citation varies across subject fields between 1.0 and 4.0, while Open Access journals in the sample show higher ratios than their non-OA counterparts. The linear correlation between GS and Scopus citation counts at the article level is high: Pearson's R is in the range of 0.8-0.9. A median Scopus indexing delay of two months compared to GS is largely though not exclusively due to missing cited references in articles in press in Scopus. The effect of double citation counts in GS due to multiple citations with identical or substantially similar meta-data occurs in less than 2 per cent of cases. Pros and cons of article-based and what is termed as concept-based citation indexes are discussed.

Modified Impact Factors

“…Since Spearman coefficient (ρ) is independent of the normality of the data distribution and moreover it can handle ties, in this study has been used Spearman ρ coefficient for the construction of the correlation matrix. 41,9 The correlation matrix (12×12) was formulated and most indicators were found to be significantly correlated (Spearman ρ > 0.8) with each other at a statistical significance level of 0.05, but there were also a few indicators that showed no strong correlation (ρ < 0.5) with the majority of other indicators. Table 5 shows that, the Spearman's ρ correlation coefficient between JCRIF and AIF (after removing self-citations) reaches a statistical significance level with a high correlation, ρ = 0.96.…”

“…It is important to understand how these indicators differ from each other, and the degree of their validity. 8,9 The Eigen factor explicitly excludes journal self-citations unlike most other indicators. 10 Reviews the literature on citation impact indicators.…”

Google Scholar como una fuente de evaluación científica: una revisión bibliográfica sobre errores de la base de datos

“…The authors analysed 982 articles, concluding that only 6.1% of them (60) had duplicate versions, which was taken to mean that they were documents that the system had not merged. Moed et al (2016) also indicated that duplicates, in the strict sense (with identical metadata), were rare (0.2%) in their study of a limited set of articles (1200) published in 12 journals. Even so, this percentage depends on the document type analysed, increasing significantly in the case of monographs.…”

“…This may have been due to the fact that the team at the beginning was made up of only two people (OrdunaMalea et al, 2016). In fact, some of the limitations or criticisms that it received in its early days, such as coverage and speed of indexation (Jacsó, 2005a), were later transformed into strengths (Moed et al, 2016;Thelwall and Kousha, 2017, in press). …”

“…These errors, even if they have been minimised, still exist. For example, Moed et al (2016) indicated that one of the mostcited articles in the Journal of Virology, according to Google Scholar (270 citations), received most of these citations (180) erroneously.…”