On Forgetting to Cite Older Papers: An Analysis of the ACL Anthology

Abstract

The field of natural language processing is experiencing a period of unprecedented growth, and with it a surge of published papers. This represents an opportunity for us to take stock of how we cite the work of other researchers, and whether this growth comes at the expense of "forgetting" about older literature. In this paper, we address this question through bibliographic analysis. We analyze the age of outgoing citations in papers published at selected ACL venues between 2010 and 2019, finding that there is indeed a tendency for recent papers to cite more recent work, but the rate at which papers older than 15 years are cited has remained relatively stable.

It’s MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Risk

“…Patterns of citation in NLP Several factors have been shown to correlate with citation rate in NLP, including author geographic location (Rungta et al, 2022), author gender (Mohammad, 2020), and publication date (Bollmann and Elliott, 2020;Singh et al, 2023). Bollmann and Elliott (2020) conduct a bibliometric anaylsis of the ACL Anthology, finding that the mean age of papers cited decreased significantly from 2010 to 2019.…”

Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023)

“…As the Anthology does not provide any data on citations, such studies require either data mining of the PDFs, a combination with data from external sources such as Google Scholar or Semantic Scholar, 27 or the use of a dataset that already provides this, like NLP Scholar (Mohammad, 2020b). Despite this necessary extra step, studies have focused on the ACL Anthology to analyze incoming citations (Mohammad, 2020a), outgoing citations (Bollmann and Elliott, 2020;Singh et al, 2023), or geographic citation gaps (Rungta et al, 2022). Van Dongen et al (2020) present a model for citation count prediction.…”

Two Decades of the ACL Anthology: Development, Impact, and Open Challenges

“…As the Anthology does not provide any data on citations, such studies require either data mining of the PDFs, a combination with data from external sources such as Google Scholar or Semantic Scholar, 27 or the use of a dataset that already provides this, like NLP Scholar (Mohammad, 2020b). Despite this necessary extra step, studies have focused on the ACL Anthology to analyze incoming citations (Mohammad, 2020a), outgoing citations (Bollmann and Elliott, 2020;Singh et al, 2023), or geographic citation gaps (Rungta et al, 2022). Van Dongen et al (2020) present a model for citation count prediction.…”