Peer review is the main institution responsible for the evaluation and gestation of scientific research. Although peer review is widely seen as vital to scientific evaluation, anecdotal evidence abounds of gatekeeping mistakes in leading journals, such as rejecting seminal contributions or accepting mediocre submissions. Systematic evidence regarding the effectiveness-or lack thereof-of scientific gatekeeping is scant, largely because access to rejected manuscripts from journals is rarely available. Using a dataset of 1,008 manuscripts submitted to three elite medical journals, we show differences in citation outcomes for articles that received different appraisals from editors and peer reviewers. Among rejected articles, desk-rejected manuscripts, deemed as unworthy of peer review by editors, received fewer citations than those sent for peer review. Among both rejected and accepted articles, manuscripts with lower scores from peer reviewers received relatively fewer citations when they were eventually published. However, hindsight reveals numerous questionable gatekeeping decisions. Of the 808 eventually published articles in our dataset, our three focal journals rejected many highly cited manuscripts, including the 14 most popular; roughly the top 2 percent. Of those 14 articles, 12 were deskrejected. This finding raises concerns regarding whether peer review is ill-suited to recognize and gestate the most impactful ideas and research. Despite this finding, results show that in our case studies, on the whole, there was value added in peer review. Editors and peer reviewers generally-but not always-made good decisions regarding the identification and promotion of quality in scientific manuscripts.peer review | innovation | decision making | publishing | creativity P eer review alters science via the filtering out of rejected manuscripts and the revision of eventually published articles. Publication in leading journals is linked to professional rewards in science, which influences the choices scientists make with their work (1). Although peer review is widely cited as central to academic evaluation (2, 3), numerous scholars have expressed concern about the effectiveness of peer review, particularly regarding the tendency to protect the scientific status quo and suppress innovative findings (4, 5). Others have focused on errors of omission in peer review, offering anecdotes of seminal scientific innovations that faced emphatic rejections from high-status gatekeepers and journals before eventually achieving publication and positive regard (6-8). Unfortunately, systematic study of peer review is difficult, largely because of the sensitive and confidential nature of the subject matter. Based on a dataset of 1,008 manuscripts submitted to three leading medical journals-Annals of Internal Medicine, British Medical Journal, and The Lancet-we analyzed the effectiveness of peer review. In our dataset, 946 submissions were rejected and 62 were accepted. Among the rejections, we identified 757 manuscripts eventually publis...

The fairness of scholarly peer review has been challenged by evidence of disparities in publication outcomes based on author demographic characteristics. To assess this, we conducted an exploratory analysis of peer review outcomes of 23,876 initial submissions and 7,192 full submissions that were submitted to the biosciences journal eLife between 2012 and 2017. Women and authors from nations outside of North America and Europe were underrepresented both as gatekeepers (editors and peer reviewers) and authors. We found evidence of a homophilic relationship between the demographics of the gatekeepers and authors and the outcome of peer review; that is, there were higher rates of acceptance in the case of gender and country homophily. The acceptance rate for manuscripts with male last authors was seven percent, or 3.5 percentage points, greater than for female last authors (95% CI = [0.5, 6.4]); this gender inequity was greatest, at nine percent or about 4.8 percentage points (95% CI = [0.3, 9.1]), when the team of reviewers was all male; this difference was smaller and not significantly different for mixed-gender reviewer teams. Homogeny between countries of the gatekeeper and the corresponding author was also associated with higher acceptance rates for many countries. To test for the persistence of these effects after controlling for potentially confounding variables, we conducted a logistic regression including document and author metadata. Disparities in acceptance rates associated with gender and country of affiliation and the homophilic associations remained. We conclude with a discussion of mechanisms that could contribute to this effect, directions for future research, and policy implications. Code and anonymized data have been made available at https://github.com/murrayds/elife-analysis 1/61 Submitted to PLOS Biology Author summaryPeer review, the primary method by which scientific work is evaluated, is ideally a fair and equitable process in which scientific work is judged solely on its own merit. However, the integrity of peer review has been called into question based on evidence that outcomes often differ between male and female authors, and for authors in different countries. We investigated such disparities at the biosciences journal eLife by analyzing the demographics of authors and gatekeepers (editors and peer reviewers), and peer review outcomes of all submissions between 2012 and 2017. Outcomes were more favorable for male authors and those affiliated with institutions in North America and Europe; these groups were also over-represented among gatekeepers. There was evidence that peer review outcomes were influenced by homophily-a preference of gatekeepers for manuscripts from authors with shared characteristics. We discuss mechanisms that could contribute to this effect, directions for future research, and policy implications. 2/61Submitted to PLOS Biology 1 Peer review is foundational to the development, gatekeeping, and dissemination of research, 2 while also underpinning the professional hiera...

Using a database of recent articles published in the field of Global Health research, we examine institutional sources of stratification in publishing access outcomes. Traditionally, the focus on inequality in scientific publishing has focused on prestige hierarchies in established print journals. This project examines stratification in contemporary publishing with a particular focus on subscription vs. various Open Access (OA) publishing options. Findings show that authors working at lower-ranked universities are more likely to publish in closed/paywalled outlets, and less likely to choose outlets that involve some sort of Article Processing Charge (APCs; gold or hybrid OA). We also analyze institutional differences and stratification in the APC costs paid in various journals. Authors affiliated with higher-ranked institutions, as well as hospitals and non-profit organizations pay relatively higher APCs for gold and hybrid OA publications. Results suggest that authors affiliated with high-ranked universities and well-funded institutions tend to have more resources to choose pay options with publishing. Our research suggests new professional hierarchies developing in contemporary publishing, where various OA publishing options are becoming increasingly prominent. Just as there is stratification in institutional representation between different types of publishing access, there is also inequality within access types.

The emergence of open access (OA) publishing has altered incentives and opportunities for academic stakeholders and publishers. These changes have yielded a variety of new economic and academic niches, including journals with questionable peer-review systems and business models, commonly dubbed "predatory publishing." Empirical analysis of Cabellʼs Journal Blacklist reveals substantial diversity in types and degrees of predatory publishing. While some blacklisted publishers produce journals with many severe violations of academic norms, "gray" journals and publishers occupy borderline or ambiguous niches between predation and legitimacy. Predation in academic publishing is not a simple binary phenomenon and should instead be perceived as a spectrum with varying types and degrees of illegitimacy. Conceptions of predation are based on overlapping evaluations of academic and economic legitimacy. High institutional status benefits publishers by reducing conflicts between-if not aligning-professional and market institutional logics, which are more likely to conflict and create illegitimacy concerns in downmarket niches. High rejection rates imbue high-status journals with value and pricing power, while low-status OA journals face "predatory" incentives to optimize revenue via low selectivity. Status influences the social acceptability of profit-seeking in academic publishing, rendering lower-status publishers vulnerable to being perceived and stigmatized as illegitimate.

Poker is a competitive, social game of skill and luck, which presents players with numerous challenging strategic and interpersonal decisions. The adaptation of poker into a game played over the internet provides the unprecedented opportunity to quantitatively analyze extremely large numbers of hands and players. This paper analyzes roughly twenty-seven million hands played online in small-stakes, medium-stakes and high-stakes games. Using PokerTracker software, statistics are generated to (a) gauge the types of strategies utilized by players (i.e. the 'strategic demography') at each level and (b) examine the various payoffs associated with different strategies at varying levels of play. The results show that competitive edges attenuate as one moves up levels, and tight-aggressive strategies--which tend to be the most remunerative--become more prevalent. Further, payoffs for different combinations of cards, varies between levels, showing how strategic payoffs are derived from competitive interactions. Smaller-stakes players also have more difficulty appropriately weighting incentive structures with frequent small gains and occasional large losses. Consequently, the relationship between winning a large proportion of hands and profitability is negative, and is strongest in small-stakes games. These variations reveal a meta-game of rationality and psychology which underlies the card game. Adopting risk-neutrality to maximize expected value, aggression and appropriate mental accounting, are cognitive burdens on players, and underpin the rationality work--reconfiguring of personal preferences and goals--players engage into be competitive, and maximize their winning and profit chances.