Can we use Google Scholar to identify highly-cited documents?

Abstract

The main objective of this paper is to empirically test whether the identification of highly-cited documents through Google Scholar is feasible and reliable. To this end, we carried out a longitudinal analysis (1950 to 2013), running a generic query (filtered only by year of publication) to minimise the effects of academic search engine optimisation. This gave us a final sample of 64,000 documents (1,000 per year). The strong correlation between a document's citations and its position in the search results (r= -0.67) led us to conclude that Google Scholar is able to identify highly-cited papers effectively. This, combined with Google Scholar's unique coverage (no restrictions on document type and source), makes the academic search engine an invaluable tool for bibliometric research relating to the identification of the most influential scientific documents. We find evidence, however, that Google Scholar ranks those documents whose language (or geographical web domain) matches with the user's interface language higher than could be expected based on citations. Nonetheless, this language effect and other factors related to the Google Scholar's operation, i.e. the proper identification of versions and the date of publication, only have an incidental impact. They do not compromise the ability of Google Scholar to identify the highly-cited papers.

Institutional Repositories and Academic Social Networks: Competition or Complement? A Study of Open Access Policy Compliance vs. ResearchGate Participation

“…Of 122,424 articles sampled in ResearchGate, Thelwall & Kousha (2015) found that 56% contained a full-text publication. A study of highly cited documents on Google Scholar revealed that ResearchGate was the second most common provider of full-text links after nih.gov (Martín-Martín, Orduña-Malea, Ayllón, & López-Cózar, 2015). Another study found that ResearchGate "was the top single source of full-text articles" of a sample of over 7,000 articles on Google Scholar, providing 10.5% of all full-text articles found, compared with 6.5% for nih.gov (Jamali & Nabavi, 2015, p. 1649.…”

Google Scholar como una fuente de evaluación científica: una revisión bibliográfica sobre errores de la base de datos

“…For example, the volume number was sometimes used to designate the publication date, as Jacsó (2010) pointed out in the case of the "Proceedings of SPIE", or the year of the latest edition of a book was mistaken for its publication date (Dilger and Müller, 2013;Martín-Martín et al, 2017). On other occasions, there was simply no date of publication (Jacsó, 2010), a fact that was also reported by Maia et al (2016), who, after analysing 2,400 documents in the area of "Strategy as Practice", noted that 15% of the documents had no publication date on Google Scholar.…”

“…Even so, this percentage depends on the document type analysed, increasing significantly in the case of monographs. Martín-Martín et al (2017) analysed the article "Mathematical Theory of Communication", for which they detected up to 165 versions that were not correctly linked.…”

“…), although the exact components of this algorithm and the weight of each variable is unknown, for industrial property reasons. However, several empirical studies have demonstrated that the number of citations received by a document is one of the key ranking factors (Beel and Gipp, 2009;Martín-Martín et al, 2017). Another essential feature of Google Scholar is that the entire process is automated, without any human intervention, from the location of documents (crawling) to the bibliographic description (metadata parsing) and the extraction of the bibliographic references (reference parsing) that are used to compute the number of citations received by each retrieved document from all other documents.…”

Has the Time Come for Preprints in Chemistry? A Perspective onto a Meaningful Change

“…The latter search engine, furthermore, effectively tracks citation of researchers who use it also for getting updated scientometric information of relevance to their own curriculum. 35 From Chemical Communications to Organic Letters, numerous prestigious journals are available to chemists to quickly publish findings of high relevance, and even in OA format when paying the article processing charge. However, the same is true for biology and physics where numerous journals, including new "express" "protocols" and "letters" versions of prestigious titles, offer a fast track to peer reviewed publication, but this has not slowed down acceptance of prepublishing in those communities.…”