Initial evidence for biased decision-making despite human-centered AI explanations

Scharowski, Nicolas; Brühlmann, Florian

doi:10.31219/osf.io/5jzmb

Cited by 2 publications

(2 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Credibility: Certification provides evidence of compliance, which increases the confidence of consumers, customers, or partners [81]. • Competitive advantage: Certified organizations can gain a competitive advantage in the marketplace as certification can be a differentiating factor.…”

Section: Certification Of Trustworthy Aismentioning

confidence: 99%

Trustworthiness Assurance Assessment for High-Risk AI-Based Systems

Stettinger,

Weissensteiner,

Khastgir

2024

IEEE Access

View full text Add to dashboard Cite

This work proposes methodologies for ensuring the trustworthiness of high-risk artificial intelligence (AI) systems (AIS) to achieve compliance with the European Union's (EU) AI Act. Highrisk classified AIS must fulfill seven requirements to be considered trustworthy and human-centric, and subsequently be considered for deployment. These requirements are equally important, mutually supportive, and should be implemented and evaluated throughout the AI lifecycle. The assurance of trustworthiness is influenced by ethical considerations, amongst others. Hence, the operational design domain (ODD) and behavior competency (BC) concepts from the automated driving domain are utilized in risk assessment strategies to quantify different types of residual risks. The methodology presented is guided by the consistent application of the ODD and its related BC concept throughout the entire AI lifecycle, focusing on the trustworthiness assurance framework and its associated process as the main pillars for AIS certification. The achievement of the overall objective of trustworthy and human-centric AIS is divided into seven interconnected sub-goals: the formulation of use restrictions, the trustworthiness assurance/argument itself, the identification of dysfunctional cases, the utilization of scenario databases and datasets, the application of metrics for evaluation, the implementation of the proposed concept across the AI lifecycle, and sufficient consideration of human factors. The role of standards in the assurance process is discussed, considering any existing gaps and areas for improvement. The work concludes with a summary of the developed approach, highlighting key takeaways and action points. Finally, a roadmap to ensure trustworthy and human-centric behavior of future AIS is outlined.

show abstract

Section: Certification Of Trustworthy Aismentioning

confidence: 99%

Trustworthiness Assurance Assessment for High-Risk AI-Based Systems

Stettinger,

Weissensteiner,

Khastgir

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Finally, in these studies, the measurement of trust is performed with a myriad of different instruments. Examples are self-assessment questionnaires [11,52,68,74,88], behavioural measures (e.g., counts of agreements with the ML model in a sequence of decisions) [41,62,70], or qualitative measures, based, for instance, on interview transcripts [9,10,17,22].…”

Section: Empirical Measurement Of Trust -Current Approachesmentioning

confidence: 99%

The Value of Measuring Trust in AI - A Socio-Technical System Perspective

Benk¹,

Tolmeijer²,

Wangenheim³

et al. 2022

Preprint

View full text Add to dashboard Cite

Building trust in AI-based systems is deemed critical for their adoption and appropriate use. Recent research has thus attempted to evaluate how various attributes of these systems affect user trust. However, limitations regarding the definition and measurement of trust in AI have hampered progress in the field, leading to results that are inconsistent or difficult to compare. In this work, we provide an overview of the main limitations in defining and measuring trust in AI. We focus on the attempt of giving trust in AI a numerical value and its utility in informing the design of real-world human-AI interactions. Taking a socio-technical system perspective on AI, we explore two distinct approaches to tackle these challenges. We provide actionable recommendations on how these approaches can be implemented in practice and inform the design of human-AI interactions. We thereby aim to provide a starting point for researchers and designers to re-evaluate the current focus on trust in AI, improving the alignment between what empirical research paradigms may offer and the expectations of real-world human-AI interactions.

show abstract

Initial evidence for biased decision-making despite human-centered AI explanations

Cited by 2 publications

References 6 publications

Trustworthiness Assurance Assessment for High-Risk AI-Based Systems

Trustworthiness Assurance Assessment for High-Risk AI-Based Systems

The Value of Measuring Trust in AI - A Socio-Technical System Perspective

Contact Info

Product

Resources

About