Fairness, explainability and in-between: understanding the impact of different explanation methods on non-expert users’ perceptions of fairness toward an algorithmic system

Tal, Avital Shulner; Kuflik, Tsvi; Kliger, Doron

doi:10.1007/s10676-022-09623-4

Cited by 30 publications

(20 citation statements)

References 33 publications

(60 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another set of studies looked at explanations for a decision as a critical aspect of perceived fairness. Explanations for an algorithmic decision significantly increased respondents' perceptions of fairness in several studies (Binns et al, 2018; Dodge et al, 2019; Shin, 2021; Shulner-Tal et al, 2022). However, the results are very nuanced.…”

Section: Resultsmentioning

confidence: 98%

See 1 more Smart Citation

Fairness perceptions of algorithmic decision-making: A systematic review of the empirical literature

et al. 2022

View full text Add to dashboard Cite

Algorithmic decision-making increasingly shapes people's daily lives. Given that such autonomous systems can cause severe harm to individuals and social groups, fairness concerns have arisen. A human-centric approach demanded by scholars and policymakers requires considering people's fairness perceptions when designing and implementing algorithmic decision-making. We provide a comprehensive, systematic literature review synthesizing the existing empirical insights on perceptions of algorithmic fairness from 58 empirical studies spanning multiple domains and scientific disciplines. Through thorough coding, we systemize the current empirical literature along four dimensions: (1) algorithmic predictors, (2) human predictors, (3) comparative effects (human decision-making vs. algorithmic decision-making), and (4) consequences of algorithmic decision-making. While we identify much heterogeneity around the theoretical concepts and empirical measurements of algorithmic fairness, the insights come almost exclusively from Western-democratic contexts. By advocating for more interdisciplinary research adopting a society-in-the-loop framework, we hope our work will contribute to fairer and more responsible algorithmic decision-making.

show abstract

Section: Resultsmentioning

confidence: 98%

“…Moreover, several studies showed that different explanation styles (e.g. case-based , sensitivity-based , demographic-based , input influence-based ) affected respondents' perceived fairness differently (Binns et al, 2018; Dodge et al, 2019; Schoeffer et al, 2021; Shulner-Tal et al, 2022).…”

Section: Resultsmentioning

confidence: 99%

Fairness perceptions of algorithmic decision-making: A systematic review of the empirical literature

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Dodge et al [10] find that people perceive global and local explanations differently, but also conclude that the effect of explanations depends on "the kinds of fairness issues and user profiles." Similarly, Shulner-Tal et al [34] found that some explanations "are more beneficial than others," but perceptions mainly depend on "the outcome of the system." B: XAI and reliance on AI Another set of works have examined how explanations may impact people's reliance on AI.…”

Section: Takeawaysmentioning

confidence: 97%

“…Prior findings are inconclusive However, as of today, there is no conclusive empirical evidence showing that explanations facilitate human-AI complementarity. Prior work has found that explanations can influence people's fairness perceptions towards AI models and their predictions in positive or negative ways (e.g., [4,10,25,34]). Other findings suggest that explanations may (e.g., [7,24]) or may not (e.g., [1,3,18,32]) lead to enhanced human-AI performance.…”

Section: Introductionmentioning

confidence: 99%

On the Relationship Between Explanations, Fairness Perceptions, and Decisions

Schoeffer¹,

De‐Arteaga²,

Kuehl³

2022

Preprint

View full text Add to dashboard Cite

It is known that recommendations of AI-based systems can be incorrect or unfair. Hence, it is often proposed that a human be the final decision-maker. Prior work has argued that explanations are an essential pathway to help human decision-makers enhance decision quality and mitigate bias, i.e., facilitate human-AI complementarity. For these benefits to materialize, explanations should enable humans to appropriately rely on AI recommendations and override the algorithmic recommendation when necessary to increase distributive fairness of decisions. The literature, however, does not provide conclusive empirical evidence as to whether explanations enable such complementarity in practice. In this work, we (a) provide a conceptual framework to articulate the relationships between explanations, fairness perceptions, reliance, and distributive fairness, (b) apply it to understand (seemingly) contradictory research findings at the intersection of explanations and fairness, and (c) derive cohesive implications for the formulation of research questions and the design of experiments.

show abstract

“… 43 See the experiments on human-in-the-loop evaluations of explanations of different styles here: (Shulner-Tal et al, 2022 ). …”

mentioning

confidence: 99%

Transparency of AI in Healthcare as a Multilayered System of Accountabilities: Between Legal Requirements and Technical Limitations

Kiseleva

Kotzinos

Hert

2022

Front. Artif. Intell.

View full text Add to dashboard Cite

The lack of transparency is one of the artificial intelligence (AI)'s fundamental challenges, but the concept of transparency might be even more opaque than AI itself. Researchers in different fields who attempt to provide the solutions to improve AI's transparency articulate different but neighboring concepts that include, besides transparency, explainability and interpretability. Yet, there is no common taxonomy neither within one field (such as data science) nor between different fields (law and data science). In certain areas like healthcare, the requirements of transparency are crucial since the decisions directly affect people's lives. In this paper, we suggest an interdisciplinary vision on how to tackle the issue of AI's transparency in healthcare, and we propose a single point of reference for both legal scholars and data scientists on transparency and related concepts. Based on the analysis of the European Union (EU) legislation and literature in computer science, we submit that transparency shall be considered the “way of thinking” and umbrella concept characterizing the process of AI's development and use. Transparency shall be achieved through a set of measures such as interpretability and explainability, communication, auditability, traceability, information provision, record-keeping, data governance and management, and documentation. This approach to deal with transparency is of general nature, but transparency measures shall be always contextualized. By analyzing transparency in the healthcare context, we submit that it shall be viewed as a system of accountabilities of involved subjects (AI developers, healthcare professionals, and patients) distributed at different layers (insider, internal, and external layers, respectively). The transparency-related accountabilities shall be built-in into the existing accountability picture which justifies the need to investigate the relevant legal frameworks. These frameworks correspond to different layers of the transparency system. The requirement of informed medical consent correlates to the external layer of transparency and the Medical Devices Framework is relevant to the insider and internal layers. We investigate the said frameworks to inform AI developers on what is already expected from them with regards to transparency. We also discover the gaps in the existing legislative frameworks concerning AI's transparency in healthcare and suggest the solutions to fill them in.

show abstract

Fairness, explainability and in-between: understanding the impact of different explanation methods on non-expert users’ perceptions of fairness toward an algorithmic system

Cited by 30 publications

References 33 publications

Fairness perceptions of algorithmic decision-making: A systematic review of the empirical literature

Fairness perceptions of algorithmic decision-making: A systematic review of the empirical literature

On the Relationship Between Explanations, Fairness Perceptions, and Decisions

Transparency of AI in Healthcare as a Multilayered System of Accountabilities: Between Legal Requirements and Technical Limitations

Contact Info

Product

Resources

About