Transparency in Algorithmic and Human Decision-Making: Is There a Double Standard?

Zerilli, John; Knott, Alistair; Maclaurin, James; Gavaghan, Colin

doi:10.1007/s13347-018-0330-6

Cited by 193 publications

(132 citation statements)

References 29 publications

Supporting

Mentioning

114

Contrasting

Unclassified

Order By: Relevance

“…McCarthy et al, ; Mittelstadt et al, ). However, information does not always lead to better acceptance, its effect in the context of automation seems to be especially complicated (Langer et al, ), and with growing automation and more complex technologies underlying this automation (e.g., deep learning algorithms), providing information can become a challenge (Mittelstadt et al, ; Zerilli, Knott, Maclaurin, & Gavaghan, ). Therefore, it is up to designers of automated tools as well as a challenge for interdisciplinary research to make these tools as controllable, social, and transparent as possible (see also the discussion on explainable artificial intelligence that is currently shaking the field of computer science; Biran & Cotton, ; Miller, Howe, & Sonenberg, ).…”

Section: Discussionmentioning

confidence: 99%

“…Alternatively, they can ignore the recommendation but are equally unable to explain why they chose to do so. In any case, hiring managers as well as applicants would be exposed to a nontransparent, nonexplainable, and nonchallengeable highly automated decision, probably leading to especially negative reactions as well as serious legal, moral and ethical issues and challenges (Zerilli et al, ).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Highly automated job interviews: Acceptance under the influence of stakes

Langer

König

Παπαθανασίου

2019

Int J Selection Assessment

106

100

View full text Add to dashboard Cite

Technological advancements allow the automation of every part of job interviews (information acquisition, information analysis, action selection, action implementation) resulting in highly automated interviews. Efficiency advantages exist, but it is unclear how people react to such interviews (and whether reactions depend on the stakes involved). Participants (N = 123) in a 2 (highly automated, videoconference) × 2 (high‐stakes, low‐stakes situation) experiment watched and assessed videos depicting a highly automated interview for high‐stakes (selection) and low‐stakes (training) situations or an equivalent videoconference interview. Automated high‐stakes interviews led to ambiguity and less perceived controllability. Additionally, highly automated interviews diminished overall acceptance through lower social presence and fairness. To conclude, people seem to react negatively to highly automated interviews and acceptance seems to vary based on the stakes. Open Practices This study was pre‐registered on the Open Science Framework (http://osf.io/hgd5r) and on AsPredicted (https://AsPredicted.org/i52c6.pdf).

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Highly automated job interviews: Acceptance under the influence of stakes

Langer

König

Παπαθανασίου

2019

Int J Selection Assessment

106

100

View full text Add to dashboard Cite

show abstract

“…1 To prevent negative outcomes and create accountable systems that individuals can trust, many have argued that we need to open up the "black box" of AI decision-making and make it more transparent (e.g., O'Neil 2016;Wachter et al 2017;Floridi et al 2018). This "opening up" will make it easier for us to understand (interpret) the functioning of the AI as well as possible to receive explanations for individual decisions (e.g., Zarsky 2016; Lepri et al 2017;Zerilli et al 2018;Binns 2018;De Laat 2018). 2 However, though a lot of interesting work has been done in the area of transparency, far less attention has been devoted to the role of transparency in terms of how those who are ultimately affected (i.e., the general public) come to perceive AI decision-making as being legitimate and worthy of acceptance.…”

Section: Introductionmentioning

confidence: 99%

Artificial intelligence, transparency, and public decision-making

Licht

2020

AI & Soc

142

View full text Add to dashboard Cite

The increasing use of Artificial Intelligence (AI) for making decisions in public affairs has sparked a lively debate on the benefits and potential harms of self-learning technologies, ranging from the hopes of fully informed and objectively taken decisions to fear for the destruction of mankind. To prevent the negative outcomes and to achieve accountable systems, many have argued that we need to open up the "black box" of AI decision-making and make it more transparent. Whereas this debate has primarily focused on how transparency can secure high-quality, fair, and reliable decisions, far less attention has been devoted to the role of transparency when it comes to how the general public come to perceive AI decision-making as legitimate and worthy of acceptance. Since relying on coercion is not only normatively problematic but also costly and highly inefficient, perceived legitimacy is fundamental to the democratic system. This paper discusses how transparency in and about AI decision-making can affect the public's perception of the legitimacy of decisions and decision-makers and produce a framework for analyzing these questions. We argue that a limited form of transparency that focuses on providing justifications for decisions has the potential to provide sufficient ground for perceived legitimacy without producing the harms full transparency would bring.

show abstract

“…For one, like many biological cognizers, the computing systems being developed in Artificial Intelligence can be viewed as information-processing systems in which inputs are systematically transformed into outputs. 9 For another, like the computers being programmed using Machine Learning, biological cognizers are opaque in the sense that we still do not know exactly why they do what they do or how they work (Zerilli et al, 2018).…”

Section: Toward a Marrian Framework For Explainable Aimentioning

confidence: 99%

“…Investigators within the Explainable AI (XAI) research program intend to ward off these consequences through the use of analytic techniques with which to render opaque computing systems transparent. 1 Although the XAI research program has already commanded significant attention (Burrell, 2016;Doran, Schulz, & Besold, 2017;Lipton, 2016;Ras, van Gerven, & Haselager, 2018;Zerilli, Knott, Maclaurin, & Gavaghan, 2018), important normative questions remain unanswered. Most fundamentally, it remains unclear how Explainable AI should explain: what is required to render opaque computing systems transparent?…”

Section: Introductionmentioning

confidence: 99%

Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence

2019

View full text Add to dashboard Cite

Many of the computing systems programmed using Machine Learning are opaque: it is difficult to know why they do what they do or how they work. The Explainable Artificial Intelligence research program aims to develop analytic techniques with which to render opaque computing systems transparent, but lacks a normative framework with which to evaluate these techniques' explanatory success. The aim of the present discussion is to develop such a framework, while paying particular attention to different stakeholders' distinct explanatory requirements. Building on an analysis of 'opacity' from philosophy of science, this framework is modeled after David Marr's influential account of explanation in cognitive science. Thus, the framework distinguishes between the different questions that might be asked about an opaque computing system, and specifies the general way in which these questions should be answered. By applying this normative framework to current techniques such as input heatmapping, featuredetector identification, and diagnostic classification, it will be possible to determine whether and to what extent the Black Box Problem can be solved.

show abstract

Transparency in Algorithmic and Human Decision-Making: Is There a Double Standard?

Cited by 193 publications

References 29 publications

Highly automated job interviews: Acceptance under the influence of stakes

Highly automated job interviews: Acceptance under the influence of stakes

Artificial intelligence, transparency, and public decision-making

Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence

Contact Info

Product

Resources

About