A framework to contest and justify algorithmic decisions

Abstract

In this paper, we argue that the possibility of contesting the results of Algorithmic Decision Systems (ADS) is a key requirement for ADS used to make decisions with a high impact on individuals. We discuss the limitations of explanations and motivate the need for better facilities to contest or justify the results of an ADS. While the goal of an explanation is to make it possible for a human being to understand, the goal of a justification is to convince that the decision is good or appropriate. To claim that a result is good, it is necessary (1) to refer to an independent definition of what a good result is (the norm) and(2) to provide evidence that the norm applies to the case. Based on these definitions, we present a challenge and justification framework including three types of norms, a proof-of-concept implementation of this framework and its application to a credit decision system.

FMEA-AI: AI fairness impact assessment using failure mode and effects analysis

“…Although not the focus of this work, perception of fairness can be modeled as a probabilistic approach to determine whether a norm (e.g., non-discrimination regulation) is breached [26]. People also perceive fairness through a reflective, justificatory process of moral reasoning [27], which typically happens only after an action is taken.…”

“…Normative approaches may also be released by government organizations as strategic plans. For example, the US National Artificial Intelligence Research and Development Strategic Plan [37] states that researchers need to improve fairness, accountability and safety in AI, including building ethical AI (p. [26][27]. Our goal in this paper is to provide specific methods of failure-assessment-based risk analysis that can help to achieve human-centered AI that is fair.…”