What AI Practitioners Say About Human-AI Trust: Its Role, Importance, and Factors That Affect It

Abstract

Trust has become a priority when designing and deploying AI-embedded systems alongside with other Human-Centered AI values, i.e. explainability, transparency, and fairness. However, due to their multifaceted and multidisciplinary nature, these terms can have various context-dependent meanings. Thus, translating these values into design can be a challenge [6]. Trust is not an exception.Understanding what Human-AI trust is and what factors affect it comes largely from controlled lab experiments or studies with prototypes of AI-embedded systems [3,7]. However, little is known about how Human-AI trust is addressed in development and deployment of real-world AI products and services. AI practitioners, people involved in different aspects of system design and deployment in the field, with roles ranging from AI developers to project managers and policy makers, can shed light on the role of Human-AI trust and what Human-AI trust factors are considered in real organizational settings. Their insights can better detail the needs, challenges, and experiences of different stakeholders when it comes to Human-AI trust.In this work-in-progress paper, we study how Human-AI trust is addressed in development and deployment of real AI systems. We conduct a series of interviews with AI practitioners who develop and deploy AI-embedded decision support systems in various risk-sensitive contexts (finance, law, management). We specifically focus on these systems, because human trust in AI is especially pertinent for them due to their potential societal impact. The interviews are part of a bigger project around AI practitioners' experiences with Human-AI trust, but in this working paper we report the preliminary findings from the first 5 interviewees (see Table 1). Specifically, we present our preliminary analysis of participants' replies to the questions regarding the role of Human-AI trust in their practices and what factors are considered when establishing it in the context of AI-assisted decision making.For the results' analysis, two independent reviewers read all the interviews at least two times and independently identified phrases of interests and codes for them, following the thematic analysis approach [1]. Together, they compared and finalized the list of selected phrases, and fine-tuned codes' formulation. By grouping the codes, the reviewers identified three major themes: 1) the role of Human-AI trust in developing and designing AI-embedded decision support systems, 2) importance of Human-AI trust in AI practitioners' work, and 3) what factors AI practitioners believe contribute to establishing trust in their systems.