Performative Power

Hardt, Moritz; Jagadeesan, Meena; Mendler-Dünner, Celestine

doi:10.48550/arxiv.2203.17232

Cited by 4 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Though not a panacea against all concerns of fairness and minority representation, we show that the multi-learner setting has many desirable characteristics compared with single-learner settings, as studied by Hashimoto et al (2018);Zhang et al (2019). Our results show that increasing the number of learners ameliorates such pitfalls, which resonates with recent work showing that monopolies have higher performative power and lead to lower individual utility (Hardt et al, 2022).…”

Section: Discussionsupporting

confidence: 89%

Multi-learner risk reduction under endogenous participation dynamics

Dean¹,

Curmei²,

Ratliff³

et al. 2022

Preprint

View full text Add to dashboard Cite

Prediction systems face exogenous and endogenous distribution shift-the world constantly changes, and the predictions the system makes change the environment in which it operates. For example, a music recommender observes exogeneous changes in the user distribution as different communities have increased access to high speed internet. If users under the age of 18 enjoy their recommendations, the proportion of the user base comprised of those under 18 may endogeneously increase. Most of the study of endogenous shifts has focused on the single decision-maker setting, where there is one learner that users either choose to use or not.This paper studies participation dynamics between sub-populations and possibly many learners. We study the behavior of systems with risk-reducing learners and sub-populations. A risk-reducing learner updates their decision upon observing a mixture distribution of the sub-populations D in such a way that it decreases the risk of the learner on that mixture. A risk reducing sub-population updates its apportionment amongst learners in a way which reduces its overall loss. Previous work on the single learner case shows that myopic risk minimization can result in high overall loss Miller et al., 2021) and representation disparity (Hashimoto et al., 2018; Zhang et al., 2019). Our work analyzes the outcomes of multiple myopic learners and market forces, often leading to better global loss and less representation disparity.

show abstract

Section: Discussionsupporting

confidence: 89%

Multi-learner risk reduction under endogenous participation dynamics

Dean¹,

Curmei²,

Ratliff³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Concretely, [MPZ21] discuss how the choice of loss function in performative prediction should balance predictive accuracy with any externalities that arise from the impacts of prediction on the observed distribution. In a different direction, [HJM22] uses performativity as a lens with which to study notions of market power in economics. As part of their analysis, they provide a decomposition of the performative risk of a classifier into terms that represent forecasting and steering.…”

Section: Performative Prediction the Performative Prediction Framewor...mentioning

confidence: 99%

“…As part of their analysis, they provide a decomposition of the performative risk of a classifier into terms that represent forecasting and steering. While we consider how the choice of loss function determines the high-level objective, [HJM22] considers how, even for a fixed loss function, the performative risk can be decomposed into terms associated with forecasting and steering.…”

Section: Performative Prediction the Performative Prediction Framewor...mentioning

confidence: 99%

Making Decisions under Outcome Performativity

Kim¹,

Perdomo²

2022

Preprint

View full text Add to dashboard Cite

Decision-makers often act in response to data-driven predictions, with the goal of achieving favorable outcomes. In such settings, predictions don't passively forecast the future; instead, predictions actively shape the distribution of outcomes they are meant to predict. This performative prediction setting [PZMH20] raises new challenges for learning "optimal" decision rules. In particular, existing solution concepts do not address the apparent tension between the goals of forecasting outcomes accurately and steering individuals to achieve desirable outcomes.To contend with this concern, we introduce a new optimality concept-performative omniprediction-adapted from the supervised (non-performative) learning setting [GKR + 22]. A performative omnipredictor is a single predictor that simultaneously encodes the optimal decision rule with respect to many possibly-competing objectives. Our main result demonstrates that efficient performative omnipredictors exist, under a natural restriction of performative prediction, which we call outcome performativity. On a technical level, our results follow by carefully generalizing the notion of outcome indistinguishability [DKR + 21] to the outcome performative setting. From an appropriate notion of Performative OI, we recover many consequences known to hold in the supervised setting, such as omniprediction and universal adaptability [KKG + 22].

show abstract

“…While the point that standard RL can lead to "feedback tampering" incentives is not new (Everitt et al, 2021b), most work so far has focused on the limitations of U RT -like objectives and on removing all influence incentives (Farquhar et al, 2022;Carroll et al, 2022;Kasirzadeh & Evans, 2023). Similarly, there has been work on training AI systems to beneficially influence humans in settings with unambiguous notions of optimality (Hong et al, 2023;Xie et al, 2020;Kim et al, 2022;Hardt et al, 2022). Instead, we study the challenges associated with choosing any notion of optimality in settings of (potentially legitimate) reward change.…”

Section: Related Workmentioning

confidence: 99%

Estimating and Penalizing Preference Shift in Recommender Systems

Carroll

Hadfield-Menell

Russell

et al. 2021

Fifteenth ACM Conference on Recommender Systems

View full text Add to dashboard Cite

Existing AI alignment approaches assume that preferences are static, which is unrealistic: our preferences change, and may even be influenced by our interactions with AI systems themselves. To clarify the consequences of incorrectly assuming static preferences, we introduce Dynamic Reward Markov Decision Processes (DR-MDPs), which explicitly model preference changes and the AI's influence on them. We show that despite its convenience, the static-preference assumption may undermine the soundness of existing alignment techniques, leading them to implicitly reward AI systems for influencing user preferences in ways users may not truly want. We then explore potential solutions. First, we offer a unifying perspective on how an agent's optimization horizon may partially help reduce undesirable AI influence. Then, we formalize different notions of AI alignment that account for preference change from the outset. Comparing the strengths and limitations of 8 such notions of alignment, we find that they all either err towards causing undesirable AI influence, or are overly risk-averse, suggesting that a straightforward solution to the problems of changing preferences may not exist. As there is no avoiding grappling with changing preferences in real-world settings, this makes it all the more important to handle these issues with care, balancing risks and capabilities. We hope our work can provide conceptual clarity and constitute a first step towards AI alignment practices which explicitly account for (and contend with) the changing and influenceable nature of human preferences.

show abstract

Performative Power

Cited by 4 publications

References 0 publications

Multi-learner risk reduction under endogenous participation dynamics

Multi-learner risk reduction under endogenous participation dynamics

Making Decisions under Outcome Performativity

Estimating and Penalizing Preference Shift in Recommender Systems

Contact Info

Product

Resources

About