Understanding Over Participation in Simple Contests

Levy, Priel; Sarne, David

doi:10.1609/aaai.v32i1.11508

Cited by 7 publications

(1 citation statement)

References 41 publications

(46 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This work also falls in the line of empirical works that study various behavioral aspects of human computation, including the impact of competitive (Levy and Sarne 2018) and impartial (Kotturi et al 2020) framing of the task on performance of the human agents. Additionally, it continues another direction of research (Kurokawa et al 2015;Xu et al 2019;Lian et al 2018;Stelmakh, Shah, and Singh 2019) that aims at improving the conference peer review.…”

Section: Introductionmentioning

confidence: 98%

A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large Conferences

Stelmakh

Shah

Singh

et al. 2021

AAAI

View full text Add to dashboard Cite

Conference peer review constitutes a human-computation process whose importance cannot be overstated: not only it identifies the best submissions for acceptance, but, ultimately, it impacts the future of the whole research area by promoting some ideas and restraining others. A surge in the number of submissions received by leading AI conferences has challenged the sustainability of the review process by increasing the burden on the pool of qualified reviewers which is growing at a much slower rate. In this work, we consider the problem of reviewer recruiting with a focus on the scarcity of qualified reviewers in large conferences. Specifically, we design a procedure for (i) recruiting reviewers from the population not typically covered by major conferences and (ii) guiding them through the reviewing pipeline. In conjunction with the ICML 2020 --- a large, top-tier machine learning conference --- we recruit a small set of reviewers through our procedure and compare their performance with the general population of ICML reviewers. Our experiment reveals that a combination of the recruiting and guiding mechanisms allows for a principled enhancement of the reviewer pool and results in reviews of superior quality compared to the conventional pool of reviews as evaluated by senior members of the program committee (meta-reviewers).

show abstract

Section: Introductionmentioning

confidence: 98%

A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large Conferences

Stelmakh

Shah

Singh

et al. 2021

AAAI

View full text Add to dashboard Cite

show abstract

Contest partitioning in binary contests

Levy,

Aumann,

Sarne

2024

Auton Agent Multi-Agent Syst

View full text Add to dashboard Cite

In this work we explore the opportunities presented by partitioning contestants in contest into disjoint groups, each competing in an independent contest, with its own prize. This, as opposed to most literature on contest design, which focuses on the setting of a single “grand” (possibly multi-stage) contest, wherein all potential contestants ultimately compete for the same prize(s), with few exceptions that do consider contest partitioning, yet with conflicting preference results concerning the optimal structure to be used. Focusing on binary contests, wherein the quality of contestants’ submissions are endogenously determined, we show that contest partitioning is indeed beneficial under some condition, e.g., whenever the number of contestants, or the prize amount, are “sufficiently large”, where the exact size requirements are a function of the partitioning cost. When partitioning does not entail any cost, we show that it is either a dominating or weakly dominating strategy, depending on the way the organizer’s expected benefit is determined. The analysis is further extended to consider partitioning where some of the sub-contests used contain a single contestant (a singleton). We conclude that contest partitioning is an avenue that contest designers can and should consider, when aiming to maximize their profit.

show abstract

Costly information providing in binary contests

Simon,

Levy,

Sarne

2024

Ann Math Artif Intell

View full text Add to dashboard Cite

Contests are commonly used as a mechanism for eliciting effort and participation in multi-agent settings. Naturally, and much like with various other mechanisms, the information provided to the agents prior to and throughout the contest fundamentally influences its outcomes. In this paper we study the problem of information providing whenever the contest organizer does not initially hold the information and obtaining it is potentially costly. As the underlying contest mechanism for our model we use the binary contest, where contestants’ strategy is captured by their decision whether or not to participate in the contest in the first place. Here, it is often the case that the contest organizer can proactively obtain and provide contestants information related to their expected performance in the contest. We provide a comprehensive equilibrium analysis of the model, showing that even when such information is costless, it is not necessarily the case that the contest organizer will prefer to obtain and provide it to all agents, let alone when the information is costly.

show abstract

Understanding Over Participation in Simple Contests

Cited by 7 publications

References 41 publications

A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large Conferences

A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large Conferences

Contest partitioning in binary contests

Costly information providing in binary contests

Contact Info

Product

Resources

About