Multiagent Evaluation Mechanisms

A, Tal; Dobson, M.; Procaccia, Ariel D.; Talgam-Cohen, Inbal; Tucker-Foltz, Jamie

doi:10.1609/aaai.v34i02.5543

Cited by 35 publications

(10 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, as in the work on strategic classification and its disparate impacts [Hardt, Megiddo, Papadimitriou, and Wootters, 2016, Hu, Immorlica, and Vaughan, 2019, Milli, Miller, Dragan, and Hardt, 2019, Alon, Dobson, Procaccia, Talgam-Cohen, and Tucker-Foltz, 2020, Braverman and Garg, 2020, we analyze how the strategic reactions of (some) applicants to a learning mechanism differentially impacts applicants.…”

Section: Related Workmentioning

confidence: 99%

Test-optional Policies: Overcoming Strategic Behavior and Informational Gaps

Zhi

Garg

2021

Preprint

View full text Add to dashboard Cite

Due to the Covid-19 pandemic, more than 500 US-based colleges and universities went "testoptional" for admissions and promised that they would not penalize applicants for not submitting test scores, part of a longer trend to rethink the role of testing in college admissions. However, it remains unclear how (and whether) a college can simultaneously use test scores for those who submit them, while not penalizing those who do not-and what that promise even means. We formalize these questions, and study how a college can overcome two challenges with optional testing: strategic applicants (when those with low test scores can pretend to not have taken the test), and informational gaps (it has more information on those who submit a test score than those who do not). We find that colleges can indeed do so, if and only if they are able to use information on who has test access and are willing to randomize admissions.

show abstract

Section: Related Workmentioning

confidence: 99%

Test-optional Policies: Overcoming Strategic Behavior and Informational Gaps

Zhi

Garg

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The principal aims to incentivize all agents to participate in the game with minimal payments. Alon et al study how to motivate multiple agents to take desirable actions using a common evaluation mechanism (Alon et al 2020). They solve multiple problems in different settings.…”

Section: Other Related Workmentioning

confidence: 99%

Optimal Common Contract with Heterogeneous Agents

Xiao

Wang

Chen

et al. 2020

AAAI

View full text Add to dashboard Cite

We consider the principal-agent problem with heterogeneous agents. Previous works assume that the principal signs independent incentive contracts with every agent to make them invest more efforts on the tasks. However, in many circumstances, these contracts need to be identical for the sake of fairness. We investigate the optimal common contract problem. To our knowledge, this is the first attempt to consider this natural and important generalization. We first show this problem is NP-complete. Then we provide a dynamic programming algorithm to compute the optimal contract in O(n2m) time, where n,m are the number of agents and actions, under the assumption that the agents' cost functions obey increasing difference property. At last, we generalize the setting such that each agent can choose to directly produce a reward in [0,1]. We provide an O(log n)-approximate algorithm for this generalization.

show abstract

“…Milly et al [7] state that the accuracy that strategic classification seeks leads to a raised bar for agents who naturally are qualified and puts a burden on them to prove themselves. Kleinberg and Raghavan [8], Haghtalab et al [9], Alon et al [10], Bechavod et al [11], Shavit et al [12], and Miller et al [13] focus on models in which the policy maker is interested in choosing a rule which incentivizes agent(s) to invest their effort into features that truly improve their qualification.…”

Section: Related Workmentioning

confidence: 99%

The Strategic Perceptron

Ahmadi¹,

Beyhaghi²,

Blum³

et al. 2020

Preprint

View full text Add to dashboard Cite

The classical Perceptron algorithm provides a simple and elegant procedure for learning a linear classifier. In each step, the algorithm observes the samples position and label and may update the current predictor accordingly. In presence of strategic agents, however, the classifier may not be able to observe the true position but a position where the agent pretends to be in order to be classified desirably. Unlike the original setting with perfect knowledge of positions, in this situation the Perceptron algorithm fails to achieve its guarantees, and we illustrate examples with the predictor oscillating between two solutions forever, never reaching a perfect classifier even though one exists. Our main contribution is providing a modified Perceptron-style algorithm which finds a classifier in presence of strategic agents with both 2 and weighted 1 manipulation costs. In our baseline model, knowledge of the manipulation costs is assumed. In our most general model, we relax this assumption and provide an algorithm which learns and refines both the classifier and its cost estimates to achieve good mistake bounds even when manipulation costs are unknown.

show abstract

Multiagent Evaluation Mechanisms

Cited by 35 publications

References 11 publications

Test-optional Policies: Overcoming Strategic Behavior and Informational Gaps

Test-optional Policies: Overcoming Strategic Behavior and Informational Gaps

Optimal Common Contract with Heterogeneous Agents

The Strategic Perceptron

Contact Info

Product

Resources

About