Trust-Aware Decision Making for Human-Robot Collaboration

Abstract

Trust in autonomy is essential for effective human-robot collaboration and user adoption of autonomous systems such as robot assistants. This paper introduces a computational model which integrates trust into robot decision-making. Specifically, we learn from data a partially observable Markov decision process (POMDP) with human trust as a latent variable. The trust-POMDP model provides a principled approach for the robot to (i) infer the trust of a human teammate through interaction, (ii) reason about the effect of its own actions on human trust, and (iii) choose actions that maximize team performance over the long term. We validated the model through human subject experiments on a table-clearing task in simulation (201 participants) and with a real robot (20 participants). In our studies, the robot builds human trust by manipulating low-risk objects first. Interestingly, the robot sometimes fails intentionally in order to modulate human trust and achieve the best team performance. These results show that the trust-POMDP calibrates trust to improve human-robot team performance over the long term. Further, they highlight that maximizing trust alone does not always lead to the best performance.:2 M. Chen et al. Fig. 1. A robot and a human collaborate to clear a table. The human, with low initial trust in the robot, intervenes to stop the robot from moving the wine glass.This study revealed that, in order to achieve fluent human-robot collaboration, the robot should monitor human trust and influence it so that it matches the system's capabilities. In our study, for instance, the robot should build human trust first by acting in a trustworthy manner, before going for the wine glass.We propose a trust-based computational model of robot decision making: Since trust is not fully observable, we model it as a latent variable in a partially observable Markov decision process (POMDP) [19]. Our trust-POMDP model contains two key components: (i) a trust dynamics model, which captures the evolution of human trust in the robot, and (ii) a human decision model, which connects trust with human actions. Our POMDP formulation can accommodate a variety of trust dynamics and human decision models. Here, we adopt a data-driven approach and learn these models from data.Although prior work has studied human trust elicitation and modeling [12,22,36,37], we close the loop between trust modeling and robot decision-making. The trust-POMDP enables the robot to systematically infer and influence the human collaborator's trust, and leverage trust for improved human-robot collaboration and long-term task performance.Consider again the table clearing example (Figure 2). The trust-POMDP strategy first removes the three plastic water bottles to build up trust and only attempts to remove the wine glass afterwards. In contrast, a baseline myopic strategy maximizes short-term task performance and does not account for human trust in choosing the robot actions. It first removes the wine glass, which offers the highest reward, resulting in unnecessary ...

The ANEMONE: Theoretical Foundations for UX Evaluation of Action and Intention Recognition in Human-Robot Interaction

“…These robots will be situated and embedded in technologically enriched environments, in which information will be exchanged ‘ naturally ’ between humans and robots, resulting in hybrid worlds where humans coexist in the digital and the real world [ 1 ]. In order to become a natural part in the daily life of humans in domestic, vocational, as well as public contexts, interaction and coexistence with several kinds of robots has to be experienced positively by humans while also being suited to our purposes, with the interaction being smooth and satisfying [ 2 , 3 , 4 , 5 ]. This means that humans should experience that a robot delivers according to existing explicit and implicit goals, which means that it performs efficiently, and in a way that makes humans feel trust, safety, and convenience while being together [ 5 ].…”

Trust in Robots: Challenges and Opportunities

“…To handle this ambiguity in definitions, we take a utilitarian approach to defining trust for HRI-we adopt a trust definition that gives us practical benefits in terms of developing appropriate robot behavior using planning and control [26,27]. As we will see over the course of this paper, this choice of definition allows us to embed the notion of trust into formal computational frameworks, specifically probabilistic graphical models [28], which in turn allows us to leverage powerful computational techniques for estimation, inference, planning, and coordination.…”

“…A more general approach is to directly model the human's dynamic trust in the robot. Work in this area has focused on two problems: (i) estimating trust based on observations of the human's behavior [18,27,[57][58][59][60][61][62][63][64] and (ii) utilizing the estimate of trust to guide robot behavior [27,58,[65][66][67][68][69].…”

Real-Time Estimation of Drivers’ Trust in Automated Driving Systems

“…Trust is fundamental to effective collaboration between humans and robotic systems [39]. Trust has been studied by the human-robot interaction (HRI) community, especially from researchers who are interested in robotic technologies acceptance and human-robot teams [8,20,39,41,51]. Researchers have been trying to understand the impacts of robots' behaviors on humans' trust evolution over time [42].…”

“…Researchers have been trying to understand the impacts of robots' behaviors on humans' trust evolution over time [42]. Moreover, they aim to use this understanding to design robots that are aware of humans' trust to operate in contexts involving collaboration with those humans [7,8]. Particularly for self-driving vehicles and automated driving systems (ADSs), trust has been used to explore consumer attitudes and enrich the discussion about safety perception [22].…”