Is Citation Count a Legitimate Indicator of Scientific Impact? A Case Study of Upper (1974) “The Unsuccessful Self-Treatment of a Case of Writer’s Block” and Its Derivatives

Abstract

The work by Upper (1974) was a blank paper. Multiple replication studies were published. This work examined the number of citations received by these papers, and manually checked the citing papers to determine why they made the citations. The Dimensions literature database was queried with the search string: (unsuccessful treatment writer’s block). The search yielded 14 articles, two of which were irrelevant and excluded. The 12 papers remained after screening included the original study by Upper (1974), nine replication studies, one review, and one meta-analysis. The original work received 43 citations, but related works had fewer than 10 citations each. One fourth of citations of Upper (1974) were being satiric on “nothing” or “precise” from papers dealing with unrelated concepts, and five citations were deemed erroneous/digressed. One citation was made to acknowledge the reviewer’s comments to Upper (1974), which did not involve Upper’s own ideas. This work exposed a scenario where there were limitations of using citation count as the only metric to gauge scientific impact of journal articles.