Stress Testing during Times of War

Judge, Kathryn

doi:10.1017/9781108903011.015

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This approach differs from the existing literature which assumes that regulators can exercise choice over how much information to request. In addition, since the result of the audit in our setting is not publicly disclosed, we abstract from questions about the design of optimal public disclosure (Goldstein and Leitner, 2017;Faria-e Castro et al, 2017;Williams, 2017;Judge, 2020).…”

Section: Introductionmentioning

confidence: 99%

Unpacking the Black Box: Regulating Algorithmic Decisions

Blattner¹,

Nelson²,

Spiess³

2021

Preprint

View full text Add to dashboard Cite

We characterize optimal oversight of algorithms in a world where an agent designs a complex prediction function but a principal is limited in the amount of information she can learn about the prediction function. We show that limiting agents to prediction functions that are simple enough to be fully transparent is inefficient as long as the bias induced by misalignment between principal's and agent's preferences is small relative to the uncertainty about the true state of the world. Algorithmic audits can improve welfare, but the gains depend on the design of the audit tools. Tools that focus on minimizing overall information loss, the focus of many post-hoc explainer tools, will generally be inefficient since they focus on explaining the average behavior of the prediction function rather than sources of mis-prediction, which matter for welfare-relevant outcomes. Targeted tools that focus on the source of incentive misalignment, e.g., excess false positives or racial disparities, can provide first-best solutions. We provide empirical support for our theoretical findings using an application in consumer lending.

show abstract

Section: Introductionmentioning

confidence: 99%