Communicating intent to develop shared situation awareness and engender trust in human-agent teams

Schaefer, Kristin E.; Straub, Edward R.; Chen, Jessie Y. C.; Putney, Joe; Evans, A. William

doi:10.1016/j.cogsys.2017.02.002

Cited by 90 publications

(54 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is vital to study trust in intelligent agents since it plays a crucial role in determining humans' perception and behavior related to technology (Lee and See, 2004). Until recently, however, the vast majority of studies about trust in computers were focused on other topics such as the influence of trust on over-reliance on the system (Van Dongen and Van Maanen, 2013), improving human-computer trust calibration (de Visser et al, 2014;McGuirl and Sarter, 2006) enhancing trust via system transparency (Koo et al, 2015;Mercado et al, 2016), promoting trust by communicating the agent's intent (Schaefer et al, 2017), and building initial trust and developing trust (Siau and Shen, 2003;Siau and Wang, 2018). Only a few studies asserted the advantages of trust repair in a technological context (Hoffman et al, 2009(Hoffman et al, , 2013.…”

Section: Trust Repair In Intelligent Agentmentioning

confidence: 99%

How should intelligent agents apologize to restore trust? Interaction effects between anthropomorphism and apology attribution on trust repair

Kim¹,

Song²

2020

Preprint

View full text Add to dashboard Cite

Trust is essential in individuals' perception, behavior, and evaluation of intelligent agents. Indeed, it is the primary motive for people to accept new technology. Thus, it is crucial to repair trust in the event when it is damaged. This study investigated how intelligent agents should apologize to recover trust and how the effectiveness of the apology is different when the agent is humanlike compared to machine-like based on two seemingly competing frameworks of the CASA (Computers-Are-Social-Actors) paradigm and automation bias. A 2 (agent: Human-like vs. Machine-like) X 2 (apology attribution: Internal vs. External) between-subject design experiment was conducted (N = 193) in the context of the stock market. Participants were presented with a scenario in which they were supposed to make investment choices with the help of an artificial intelligence agent's advice. To see the trajectory of initial trust-building, trust violation, and trust repair process, we designed an investment game that consists of 5 rounds of 8 investment choices (in total, 40 investment choices). The results show that trust was repaired more efficiently when a human-like agent apologizes with internal compared to external attribution. However, the opposite pattern was observed among participants who had machine-like agents; the external compared to internal attribution condition showed better trust repair. Both theoretical and practical implications are discussed.

show abstract

Section: Trust Repair In Intelligent Agentmentioning

confidence: 99%

How should intelligent agents apologize to restore trust? Interaction effects between anthropomorphism and apology attribution on trust repair

Kim¹,

Song²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Finally, we proposed a methodology for identifying differences among solutions to spatial planning problems with the ultimate goal of identifying the types of information (i.e., criterion) that would facilitate bidirectional communication between team members. Transparency in reasoning and decision-making can improve situation awareness among members in a team [24,90,91] but only when that information is relevant and accessible. Route planning IAs typically employ algorithms that are somewhat difficult to explain to users, which makes directly communicating the IA's reasoning process difficult.…”

Section: Joint Discussion and Conclusionmentioning

confidence: 99%

“…In human teams, teammates may develop shared mental models 1 that can facilitate prediction of team member actions, support adapting decisions based on team demands, and coordinate actions especially in circumstances where communication is limited [21]. Trust and transparency research suggests that trust can be retained if team members understand underlying intent [22][23][24]. The difficulty in understanding and communicating intent is that humans and IAs trained by machine learning may not have declarative access to the reasons they choose a particular route or action.…”

mentioning

confidence: 99%

Where Do You Think You're Going?

Perelman

Evans

Schaefer

2020

J. Hum.-Robot Interact.

Self Cite

View full text Add to dashboard Cite

Route planning is a critical behavior for human-intelligent agent (H-IA) team mobility. The scientific community has made major advances in improving route planner optimality and speed. However, human factors, such as the ability to predict and understand teammates' actions and goals, are necessary for trust development in H-IA teams. Trust is especially critical when agents' behaviors do not match human team members' expectations, or the human cannot understand the agent's underlying reasoning process. To address this issue, the artificial intelligence community has pushed toward creating human-like agent behaviors using machine learning. The problem with this approach is that we do not yet have a clear understanding of what constitutes human-like behavior across the breadth of tasks that H-IA teams undertake. This article describes an investigation and comparison of human and agent route planning behaviors, the interplay between humans and agents in collaborative planning, and the role of trust in this collaborative process. Finally, we propose a data-driven methodology for characterizing and visualizing differences among routes planned by humans and agents. This methodology provides a means to advance compatible mental model metrics and theory by informing targeted transparency manipulations, thereby improving the speed and quality of routes produced by H-IA teams. CCS Concepts: • Human-centered computing → Human computer interaction (HCI); • Computing methodologies → Planning and scheduling; • Computer systems organization → Embedded and cyber-physical systems; Robotics;

show abstract

“…Further, tasks involving close proximity teamwork may require more detailed knowledge of how the robot will act both before and during the motion, such as in collaborative furniture assembly [27] and co-located teleoperation [35]. Other related works have used turn and display indicators on the robot to communicate navigational intent [36,7,28]. These techniques were found to improve human trust and confidence in robot actions; however, they did not show a significant improvement in communicating high-fidelity navigational intent due to an inability to express high detail in the motion plan.…”

Section: Related Workmentioning

confidence: 99%

Communicating Robot Arm Motion Intent Through Mixed Reality Head-Mounted Displays

Rosen

Whitney

Phillips

et al. 2019

Springer Proceedings in Advanced Robotics

View full text Add to dashboard Cite

Efficient motion intent communication is necessary for safe and collaborative work environments with collocated humans and robots. Humans efficiently communicate their motion intent to other humans through gestures, gaze, and social cues. However, robots often have difficulty efficiently communicating their motion intent to humans via these methods. Many existing methods for robot motion intent communication rely on 2D displays, which require the human to continually pause their work and check a visualization. We propose a mixed reality head-mounted display visualization of the proposed robot motion over the wearer's real-world view of the robot and its environment. To evaluate the effectiveness of this system against a 2D display visualization and against no visualization, we asked 32 participants to labeled different robot arm motions as either colliding or non-colliding with blocks on a table. We found a 16% increase in accuracy with a 62% decrease in the time it took to complete the task compared to the next best system. This demonstrates that a mixed-reality HMD allows a human to more quickly and accurately tell where the robot is going to move than the compared baselines.

show abstract

Communicating intent to develop shared situation awareness and engender trust in human-agent teams

Cited by 90 publications

References 45 publications

How should intelligent agents apologize to restore trust? Interaction effects between anthropomorphism and apology attribution on trust repair

How should intelligent agents apologize to restore trust? Interaction effects between anthropomorphism and apology attribution on trust repair

Where Do You Think You're Going?

Communicating Robot Arm Motion Intent Through Mixed Reality Head-Mounted Displays

Contact Info

Product

Resources

About