Engineering Trust in Complex Automated Systems

Lyons, Joseph B.; Koltai, Kolina; Ho, Nhut; Johnson, Walter B.; Smith, David E.; Shively, Robert J.

doi:10.1177/1064804615611272

Cited by 51 publications

(31 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accumulated research suggests that trust in automation is affected by system characteristics such as reliability degree of automation [16], [17], and transparency [10], [18], while individual characteristics such as self confidence in performing a task [19] culture [20], and environmental characteristics such as task load [21], [22] and risk [22] may affect either trust, dependence, or both.…”

Section: A Trust In Automationmentioning

confidence: 99%

See 1 more Smart Citation

Influence of Culture, Transparency, Trust, and Degree of Automation on Automation Use

Chien

Lewis

Sycara

et al. 2020

IEEE Trans. Human-Mach. Syst.

View full text Add to dashboard Cite

The reported study compares groups of 120 participants each, from the United States, Taiwan, and Turkey interacting with versions of an automated path planner that vary in transparency and degree of automation. The nationalities were selected in accordance with the theory of Cultural Syndromes as representatives of Dignity (US), Face (Taiwan), and Honor (Turkey) cultures and were predicted to differ in readiness to trust automation, degree of transparency required to use automation, and willingness to use systems with high degrees of automation. Three experimental conditions were tested. In the first, highlight, path conflicts were highlighted leaving rerouting to the participant. In the second, re-planner made requests for permission to reroute when a path conflict was detected. The third combined condition increased transparency of the re-planner by combining highlighting with rerouting to make the conflict on which decision was based visible to the user. A novel framework relating transparency, stages of automation, and trust in automation is proposed in which transparency plays a primary role in decisions to use automation but is supplemented by trust where there is insufficient information otherwise. Hypothesized cultural effects and framework predictions were confirmed.

show abstract

Section: A Trust In Automationmentioning

confidence: 99%

“…Annotating decisions with confidence judgments is another widely used technique for providing greater insight into system performance. This has been done variously by providing probabilities of success [10], [28], confidence in detection [28] F o r P e e r R e v i e w or confidence (Q values) in selected robot actions [11].…”

Section: Transparencymentioning

confidence: 99%

Influence of Culture, Transparency, Trust, and Degree of Automation on Automation Use

Chien

Lewis

Sycara

et al. 2020

IEEE Trans. Human-Mach. Syst.

View full text Add to dashboard Cite

show abstract

“…Research has shown that transparency methods in the form of decision rationale can increase trust for recommender systems in commercial aviation [7]. A replication of Lyons and colleagues [7] using a high-delity simulation found that added rationale increases user trust and reliance on the decision aid while reducing veri cation (i.e., second guessing) of the automation's recommendation [8]. Humans need to understand the logic behind any recommendations by a complex system.…”

Section: Trust and Transparency In Learning Systemsmentioning

confidence: 99%

Trust between Humans and Learning Machines: Developing the Gray Box

Christensen

Lyons

2017

Mechanical Engineering

Self Cite

View full text Add to dashboard Cite

This article explores the notion of the ‘Gray Box’ to symbolize the idea of providing sufficient information about the learning technology to establish trust. The term system is used throughout this article to represent an intelligent agent, robot, or other form of automation that possesses both decision initiative and authority to act. The article also discusses a proposed and tested Situation Awareness-based Agent Transparency (SAT) model, which posits that users need to understand the system’s perception, comprehension, and projection of a situation. One of the key challenges is that a learning system may adopt behavior that is difficult to understand and challenging to condense to traditional if-then statements. Without a shared semantic space, the system will have little basis for communicating with the human. One of the key recommendations of this article is that there is a need to provide learning systems with transparency as to the state of the human operator, including their momentary capabilities and potential impact of changes in task allocation and teaming approach.

show abstract

“…Willingness to use automated or autonomous systems is highly related to trust, which may be influenced by various factors, such as system reliability, human predisposition to trust, prior and current experience, and many others. A growing literature suggests that trust significantly contributes to human decisions about the use of automation [1][2][3][4]. In other words, people tend to rely on automation they trust and not use automation they do not trust.…”

Section: Introductionmentioning

confidence: 99%

The Effect of Culture on Trust in Automation

Chien

Lewis

Sycara

et al. 2018

ACM Trans. Interact. Intell. Syst.

View full text Add to dashboard Cite

Trust in automation has become a topic of intensive study since the late 1990s and is of increasing importance with the advent of intelligent interacting systems. While the earliest trust experiments involved human interventions to correct failures/errors in automated control systems, a majority of subsequent studies have investigated information acquisition and analysis decision aiding tasks such as target detection for which automation reliability is more easily manipulated. Despite the high level of international dependence on automation in industry, almost all current studies have employed Western samples primarily from the U.S. The present study addresses these gaps by running a large sample experiment in three (U.S., Taiwan, and Turkey) diverse cultures using a "trust sensitive task" consisting of both automated control and target detection subtasks. This article presents results for the target detection subtask for which reliability and task load were manipulated. The current experiments allow us to determine whether reported effects are universal or specific to Western culture, vary in baseline or magnitude, or differ across cultures. Results generally confirm consistent effects of manipulations across the three cultures as well as cultural differences in initial trust and variation in effects of manipulations consistent with 10 cultural hypotheses based on Hofstede's Cultural Dimensions and Leung and Cohen's theory of Cultural Syndromes. These results provide critical implications and insights for correct trust calibration and to enhance human trust in intelligent automation systems across cultures. Additionally, our results would be useful in designing intelligent systems for users of different cultures. Our article presents the following contributions: First, to the best of our knowledge, this is the first set of studies that deal with cultural factors across all the cultural syndromes identified in the literature by comparing trust in the Honor, Face, Dignity cultures. Second, this is the first set of studies that uses a validated cross-cultural trust measure for measuring trust in automation. Third, our experiments are the first to study the dynamics of trust across cultures.

show abstract

Engineering Trust in Complex Automated Systems

Cited by 51 publications

References 9 publications

Influence of Culture, Transparency, Trust, and Degree of Automation on Automation Use

Influence of Culture, Transparency, Trust, and Degree of Automation on Automation Use

Trust between Humans and Learning Machines: Developing the Gray Box

The Effect of Culture on Trust in Automation

Contact Info

Product

Resources

About