Inferring Trust From Users’ Behaviours; Agents’ Predictability Positively Affects Trust, Task Performance and Cognitive Load in Human-Agent Real-Time Collaboration

Abstract

Collaborative virtual agents help human operators to perform tasks in real-time. For this collaboration to be effective, human operators must appropriately trust the agent(s) they are interacting with. Multiple factors influence trust, such as the context of interaction, prior experiences with automated systems and the quality of the help offered by agents in terms of its transparency and performance. Most of the literature on trust in automation identified the performance of the agent as a key factor influencing trust. However, other work has shown that the behavior of the agent, type of the agent’s errors, and predictability of the agent’s actions can influence the likelihood of the user’s reliance on the agent and efficiency of tasks completion. Our work focuses on how agents’ predictability affects cognitive load, performance and users’ trust in a real-time human-agent collaborative task. We used an interactive aiming task where participants had to collaborate with different agents that varied in terms of their predictability and performance. This setup uses behavioral information (such as task performance and reliance on the agent) as well as standardized survey instruments to estimate participants’ reported trust in the agent, cognitive load and perception of task difficulty. Thirty participants took part in our lab-based study. Our results showed that agents with more predictable behaviors have a more positive impact on task performance, reliance and trust while reducing cognitive workload. In addition, we investigated the human-agent trust relationship by creating models that could predict participants’ trust ratings using interaction data. We found that we could reliably estimate participants’ reported trust in the agents using information related to performance, task difficulty and reliance. This study provides insights on behavioral factors that are the most meaningful to anticipate complacent or distrusting attitudes toward automation. With this work, we seek to pave the way for the development of trust-aware agents capable of responding more appropriately to users by being able to monitor components of the human-agent relationships that are the most salient for trust calibration.

Mapping the Design Space of Human-AI Interaction in Text Summarization

“…Therefore, they desired to adjust the extent of AIpredicted updates based on their intention. Echoing literature on predictable AI systems (Daronnat et al, 2021), future interactive editing systems should consider user expectations and empower users to preview AI actions. Systems could also model human editing intention, perhaps via action history like number and location of edits, and adapt AI actions accordingly to better serve user goals.…”

Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions

“…Screening of dangerous objects/subjects [4, 14, 15, 19, 36, 51, 54-56, 85, 94, 107, 117], detection of system malfunctions [89,90], crime prevention [10], recidivism prediction [132], watch a video of a house search [66,83] Transportation Responding to take-over requests [2, 6, 7, 47, 61, 68-70, 77, 84, 98], collusion avoidance [8], managing (air) traffic [31,118], pedestrians interaction with AV [48], observe AVs [64,119], drive in driving simulator [87,96,136] Military Screening tasks [17,33,44,63,81,82,127,130,135,139,140], gathering of information [58], mission planning [92], human-AI collaboration for search and destroy missions [116] Production Improving production [73,133,143], disassembly [5], moving objects [34,35,45], demand forecasting [39], harvesting [113], quality checks [142] Gaming trust game [3,23], collaboration game [24], flanker task…”

“…Drawing on the definition of trust, collaborative tasks should require less trust because the individual autonomy is greater and the dependency on another entity is less, reducing uncertainty and risk. However, need for trust in collaborative tasks might increase for situations of human-AI teaming, where the human team member is dependent on the performance of the system, such as [18,24,116] report. Such situations are characterized by increased dependency, possibly leading to an increased need for trust.…”

“…While the development of many complex automated technologies is driven by artificial intelligence, we cannot make specific claims about these systems. For example, some papers examined robots (see e.g., [3,5,6,22,24,50,130]). From previous work, we know that robots are increasingly equipped with artificial intelligence (see e.g., [111]).…”

Trust in Robot–Robot Scaffolding

“…Daronnat et al [18] addressed the link between the virtual agent's predictability and its effect on human trust. The assigned task in this study is a collaborative missile command game played with five agents with different performance and predictability skills.…”