Transparency, Detection and Imitation in Strategic Classification

Abstract

Given the ubiquity of AI-based decisions that affect individuals’ lives, providing transparent explanations about algorithms is ethically sound and often legally mandatory. How do individuals strategically adapt following explanations? What are the consequences of adaptation for algorithmic accuracy? We simulate the interplay between explanations shared by an Institution (e.g. a bank) and the dynamics of strategic adaptation by Individuals reacting to such feedback. Our model identifies key aspects related to strategic adaptation and the challenges that an institution could face as it attempts to provide explanations. Resorting to an agent-based approach, our model scrutinizes: i) the impact of transparency in explanations, ii) the interaction between faking behavior and detection capacity and iii) the role of behavior imitation. We find that the risks of transparent explanations are alleviated if effective methods to detect faking behaviors are in place. Furthermore, we observe that behavioral imitation --- as often happens across societies --- can alleviate malicious adaptation and contribute to accuracy, even after transparent explanations.

Prosocial dynamics in multiagent systems

“…In another direction, the way individuals strategically adapt to algorithms might depend on information collected from peers and from online platforms (Ghalme et al 2021;Bechavod et al 2022;Barsotti, Koçer, & Santos 2022). Disclosing truthful information for this purpose entails a second-order social dilemma, just as the challenge of costly reputation sharing previously discussed: individuals are required to spend time and effort (i.e., spend a cost) to offer others valuable information about their experiences, which hopefully contribute to others' possibility of algorithmic recourse (Karimi et al 2022).…”

“…The study of prosociality in large populations of adaptive agents can also be informative in the context of strategic classification. When subject to the results of an algorithmic decision, individuals can choose to improve their condition—thereby incurring high effort to improve their chances of future success—or choose to game the system—for example, by providing false information or strategically adapt features in ways that do not cause future success (Kleinberg & Raghavan 2020; Miller, Milli, & Hardt 2020; Barsotti, Koçer, & Santos 2022). Improving means that individuals are required to pay a high cost to adapt and thereby concede classifiers the benefit of keeping high accuracy.…”