To ArXiv or not to ArXiv: A Study Quantifying Pros and Cons of Posting Preprints Online

Abstract

Double-blind conferences have engaged in debates over whether to allow authors to post their papers online on arXiv or elsewhere during the review process. Independently, some authors of research papers face the dilemma of whether to put their papers on arXiv due to its pros and cons. We conduct a study to substantiate this debate and dilemma via quantitative measurements. Specifically, we conducted surveys of reviewers in two top-tier double-blind computer science conferences-ICML 2021 (5361 submissions and 4699 reviewers) and EC 2021 (498 submissions and 190 reviewers). Our two main findings are as follows. First, more than a third of the reviewers self-report searching online for a paper they are assigned to review. Second, outside the review process, we find that preprints from better-ranked affiliations see a weakly higher visibility, with a correlation of 0.06 in ICML and 0.05 in EC. In particular, papers associated with the top-10-ranked affiliations had a visibility of approximately 11% in ICML and 22% in EC, whereas the remaining papers had a visibility of 7% and 18% respectively.

Erros comuns de docentes sem letramento em Inteligência Artificial: uma revisão integrativa para o ensino superior

“…A base ArXiv.Org, de propriedade da estadunidense Cornell University é um exemplo: em vigor desde 1991, ela foi a primeira base preprint a compartilhar atualizações nas áreas de ciências da computação, física e matemática. Seu funcionamento como repositório confiável é garantido por uma junta de moderadores que revê as submissões e, nestes tempos de evolução exponencial das ciência da IA, tornou-se um novo modelo de moderação por pares (Rastogi et al, 2022).…”

How do authors’ perceptions of their papers compare with co-authors’ perceptions and peer-review decisions?

“…Within the machine learning community, [31] survey reviewers about visibility of papers submitted to a conference that anonymizes authors, and intentionally searching online for assigned papers. Or current work contributes to a tradition in machine learning venues of experimentation aimed at understanding and improving the peer-review process [3,5,16,[32][33][34][35][36][37][38].…”

“…The survey by [40] in the CHI 2011 conference had a response rate of 16%. [31] conduct multiple surveys: an anonymous survey in the ICML 2021 and EC 2021 conferences had response rates of 16% and 51% respectively; a second, non-anonymous opt-in survey in EC 2021 had a response rate of 55.78%. [17] survey authors of accepted papers in 56 computer systems conferences, with response rates ranging from 0% to 59% across these conferences.…”