Building Second-Order Mental Models for Human-Robot Interaction

Brooks, Connor; Szafir, Daniel

doi:10.48550/arxiv.1909.06508

“…For teamwork of this sort to succeed when complex tasks are at stake, humans and robots might sometimes need the capacity of theory of mind (or second-order "mental" models) to represent each other's epistemic states (knowledge, belief) and pro-attitudes (desires, goals). Theory of mind comes "live" in the human brain at age three to five (Southgate, 2013;Wellman, Cross, & Watson, 2001) and its role in cooperative human-robot interaction has received considerable attention recently (e.g., Brooks & Szafir, 2019;Devin & Alami, 2016;Görür, Rosman, Hoffman, & Albayrak, 2017;Leyzberg, Spaulding, & Scassellati, 2014;Scassellati, 2002;Zhao, Holtzen, Gao, & Zhu, 2015; for a review, see Tabrez, Luebbers, & Hayes, 2020; for implementations in "moral algorithms," see Tolmeijer, Kneer, Sarasua, Christen, & Bernstein, 2020).…”

Section: Introductionmentioning

confidence: 99%

Can a Robot Lie? Exploring the Folk Concept of Lying as Applied to Artificial Agents

Kneer

¹

2021

View full text Add to dashboard Cite

The potential capacity for robots to deceive has received considerable attention recently. Many papers explore the technical possibility for a robot to engage in deception for beneficial purposes (e.g., in education or health). In this short experimental paper, I focus on a more paradigmatic case: robot lying (lying being the textbook example of deception) for nonbeneficial purposes as judged from the human point of view. More precisely, I present an empirical experiment that investigates the following three questions: (a) Are ordinary people willing to ascribe deceptive intentions to artificial agents? (b) Are they as willing to judge a robot lie as a lie as they would be when human agents engage in verbal deception? (c) Do people blame a lying artificial agent to the same extent as a lying human agent? The response to all three questions is a resounding yes. This, I argue, implies that robot deception and its normative consequences deserve considerably more attention than they presently receive.

show abstract

“…This underlines the importance of developing systems that can understand the mental model of their users [7,19]. In fact, this strategic behaviour of the user can also leak information about the user's goal which the system could capture to further improve the optimization [6].…”

Section: Discussionmentioning

confidence: 99%

Human Strategic Steering Improves Performance of Interactive Optimization

Colella

¹

,

Daee

²

,

Jokinen

³

et al. 2020

Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization

View full text Add to dashboard Cite

A central concern in an interactive intelligent system is optimization of its actions, to be maximally helpful to its human user. In recommender systems for instance, the action is to choose what to recommend, and the optimization task is to recommend items the user prefers. The optimization is done based on earlier user's feedback (e.g. "likes" and "dislikes"), and the algorithms assume the feedback to be faithful. That is, when the user clicks like, they actually prefer the item. We argue that this fundamental assumption can be extensively violated by human users, who are not passive feedback sources. Instead, they are in control, actively steering the system towards their goal. To verify this hypothesis, that humans steer and are able to improve performance by steering, we designed a function optimization task where a human and an optimization algorithm collaborate to find the maximum of a 1-dimensional function. At each iteration, the optimization algorithm queries the user for the value of a hidden function f at a point x, and the user, who sees the hidden function, provides an answer about f (x). Our study on 21 participants shows that users who understand how the optimization works, strategically provide biased answers (answers not equal to f (x)), which results in the algorithm finding the optimum significantly faster. Our work highlights that next-generation intelligent systems will need user models capable of helping users who steer systems to pursue their goals.

show abstract

“…For teamwork of this sort to succeed when complex tasks are at stake, humans and robots might sometimes need the capacity of theory of mind (or second-order "mental" models) to represent each other's epistemic states (knowledge, belief) and pro-attitudes (desires, goals). Theory of mind comes "live" in the human brain at age three to five (Southgate, 2013;Wellman, Cross, & Watson, 2001) and its role in cooperative human-robot interaction has received considerable attention recently (e.g., Brooks & Szafir, 2019;Devin & Alami, 2016;Görür, Rosman, Hoffman, & Albayrak, 2017;Leyzberg, Spaulding, & Scassellati, 2014;Scassellati, 2002;Zhao, Holtzen, Gao, & Zhu, 2015; for a review, see Tabrez, Luebbers, & Hayes, 2020; for implementations in "moral algorithms," see Tolmeijer, Kneer, Sarasua, Christen, & Bernstein, 2020).…”

Section: Introductionmentioning

confidence: 99%