Unpacking the Black Box: Regulating Algorithmic Decisions

Abstract

We characterize optimal oversight of algorithms in a world where an agent designs a complex prediction function but a principal is limited in the amount of information she can learn about the prediction function. We show that limiting agents to prediction functions that are simple enough to be fully transparent is inefficient as long as the bias induced by misalignment between principal's and agent's preferences is small relative to the uncertainty about the true state of the world. Algorithmic audits can improve welfare, but the gains depend on the design of the audit tools. Tools that focus on minimizing overall information loss, the focus of many post-hoc explainer tools, will generally be inefficient since they focus on explaining the average behavior of the prediction function rather than sources of mis-prediction, which matter for welfare-relevant outcomes. Targeted tools that focus on the source of incentive misalignment, e.g., excess false positives or racial disparities, can provide first-best solutions. We provide empirical support for our theoretical findings using an application in consumer lending.

How do Humans Interact with Algorithms? Experimental Evidence from Health Insurance

“…Our study extends early research in decision making that examines conceptually how "recommender systems" could be used to assist consumers in on-line shopping (reviewed extensively in Adomavicius and Tuzhilin, 2005). 1 It also contributes to the nascent interdisciplinary literature on how humans and algorithms interact in a variety of decision-making settings (Luo et al, 2019;Lambrecht and Tucker, 2019;Gruber et al, 2020;Bogert et al, 2021;Blattner et al, 2021). Glikson and Woolley (2020) and Puntoni et al (2021) provide overviews.…”

Algorithmic Decision Making Methods for Fair Credit Scoring

“…This business reality raises questions regarding the supervision of such automated decisions. Blattner et al [2] discussed how algorithmic governance faces a trade-off between complexity (read performance) and oversight (read: capacity to audit). The interpretability of the models, particularly the credit scoring methods, has long been a source of concern for industry regulators (e.g., national banks, financial authorities, and governments).…”

Know Your Customer: Informed Trading by Banks

“…Finally, we contribute to a recent literature on data-driven advances in financial regulation and supervision (Spatt (2020)). For example, Blattner et al (2021) characterize optimal financial regulation when lending decisions are made by complex algorithms. Davis et al (2022) create machine learning models that regulators can use to forecast credit risk.…”