The mismeasure of science: Citation analysis

MacRoberts, Michael H.; MacRoberts, Barbara R.

doi:10.1002/asi.23970

Cited by 72 publications

(49 citation statements)

References 73 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Today, citation-based indicators are widely used and play a central role in the evaluation of scientific works. Despite their de facto use as proxies of scientific quality, citations are not able to fully capture the use and influence of scientific papers (Moed, 2005;MacRoberts & MacRoberts, 2017). Bibliometric research has also shown that a variety of factors can influence citation counts (Opthof & Leydesdorff, 2010;Waltman et al, 2011;Larivière & Gingras, 2011).…”

Section: Effect Of Document Characteristics On Citation Impactmentioning

confidence: 99%

On the relationships between bibliographic characteristics of scientific documents and citation and Mendeley readership counts: A large-scale analysis of Web of Science publications

Zahedi

Haustein

2018

Journal of Informetrics

View full text Add to dashboard Cite

In this paper we present a first large-scale analysis of the relationship between Mendeley readership and citation counts with particular documents' bibliographic characteristics. A data set of 1.3 million publications from different fields published in journals covered by the Web of Science (WoS) has been analyzed. This work reveals that document types that are often excluded from citation analysis due to their lower citation values, like editorial materials, letters, or news items, are strongly covered and saved in Mendeley, suggesting that Mendeley readership can reliably inform the analysis of these document types. Findings show that collaborative papers are frequently saved in Mendeley, which is similar to what is observed for citations. The relationship between readership and the length of titles and number of pages, however, is weaker than for the same relationship observed for citations. The analysis of different disciplines also points to different patterns in the relationship between several document characteristics, readership, and citation counts. Overall, results highlight that although disciplinary differences exist, readership counts are related to similar bibliographic characteristics as those related to citation counts, reinforcing the idea that Mendeley readership and citations capture a similar concept of impact, although they cannot be considered as equivalent indicators.

show abstract

Section: Effect Of Document Characteristics On Citation Impactmentioning

confidence: 99%

On the relationships between bibliographic characteristics of scientific documents and citation and Mendeley readership counts: A large-scale analysis of Web of Science publications

Zahedi

Haustein

2018

Journal of Informetrics

View full text Add to dashboard Cite

show abstract

“…However, weaknesses of citation indexes include that they do not take into account the reasons the article is cited (e.g. as criticism), there may be long delays from publication to placement in citation databases [50], and most importantly, they do not show the impact of the article beyond the academic research community [51,52].…”

Section: Altmetrics and The Importance Of Gaining Public Attentionmentioning

confidence: 99%

Relationship between persuasive metadiscoursal devices in research article abstracts and their attention on social media

Ngai

Singh

2020

PLoS ONE

View full text Add to dashboard Cite

Research article abstracts often convince readers that the article is worth reading. Therefore, they rely not only on the quality of arguments or novelty of findings to persuade readers but also linguistic markers in the form of metadiscourse to assert a position on an issue, increase readability of a text, engage readers, and avoid objection to the writer's interpretations, thereby enhancing the credibility of the text. Given that research article abstracts are often published online and their newsworthiness would affect whether they would be ultimately read, Altmetric.com, which emerged in 2010, can help quantify the popularity of research article abstracts by counting views on social media and other platforms such as news and policy documents. Yet a study on how metadiscoursal devices are used to persuade readers, and how they are correlated with Altmetric Attention Score (AAS) provided by Altmetric. com, merits attention. In our study, we examined 241 abstracts from 50 top journals in 12 disciplines with the highest AAS from 2014-2018 and performed a quantitative analysis of the interactive and interactional metadiscourse markers exhibited in the abstracts. Overall, we found a positive correlation between the use of metadiscourse and AAS. Furthermore, we noticed that each discipline used distinct metadiscourse markers in abstracts with high AAS, which contributed to its respective discipline-specific conventions. It has been previously shown that the use of an array of interactive and interactional metadiscourse renders the abstract more worthy of attention. Being knowledgeable of rhetorical choices in relation to metadiscoursal devices will enable writers to construct more persuasive abstracts by making informed judgments about the appropriate use of metadiscourse to draw the attention of readers in their respective disciplines.

show abstract

“…Despite its frequent use, the method is often criticized. According to Macilwain (2013), a fundamental defect of quantitative research-assessment tools, such as bibliometrics, is that they are "largely built on sand" (p. 255), which means that they cannot directly measure the 'quality of research' and instead use "weak surrogates, such as the citation indices of individuals" (p. 255). An overview of other critical points on bibliometrics (especially citation analysis) can be found in the paper The mismeasure of science: citation analysis, published by Mac-Roberts and MacRoberts (2017).…”

Section: How the Study Of Heuristics Can Inform The Study Of Bibliomementioning

confidence: 99%

Heuristics as conceptual lens for understanding and studying the usage of bibliometrics in research evaluation

Bornmann

Marewski

2019

Scientometrics

View full text Add to dashboard Cite

While bibliometrics are widely used for research evaluation purposes, a common theoretical framework for conceptually understanding, empirically studying, and effectively teaching its usage is lacking. In this paper, we outline such a framework: the fast-and-frugal heuristics research program, proposed originally in the context of the cognitive and decision sciences, lends itself particularly well for understanding and investigating the usage of bibliometrics in research evaluations. Such evaluations represent judgments under uncertainty in which typically not all possible options, their consequences, and those consequences' probabilities of occurring may be known. In these situations of incomplete information, candidate descriptive and prescriptive models of human behavior are heuristics. Heuristics are simple strategies that, by exploiting the structure of environments, can aid people to make smart decisions. Relying on heuristics does not mean trading off accuracy against effort: while reducing complexity, heuristics can yield better decisions than more information-greedy procedures in many decision environments. The prescriptive power of heuristics is documented in a cross-disciplinary literature, cutting across medicine, crime, business, sports, and other domains. We outline the fast-and-frugal heuristics research program, provide examples of past empirical work on heuristics outside the field of bibliometrics, explain why heuristics may be especially suitable for studying the usage of bibliometrics, and propose a corresponding conceptual framework.

show abstract

The mismeasure of science: Citation analysis

Cited by 72 publications

References 73 publications

On the relationships between bibliographic characteristics of scientific documents and citation and Mendeley readership counts: A large-scale analysis of Web of Science publications

On the relationships between bibliographic characteristics of scientific documents and citation and Mendeley readership counts: A large-scale analysis of Web of Science publications

Relationship between persuasive metadiscoursal devices in research article abstracts and their attention on social media

Heuristics as conceptual lens for understanding and studying the usage of bibliometrics in research evaluation

Contact Info

Product

Resources

About