Transparency in a Human-Machine Context: Approaches for Fostering Shared Awareness/Intent

Abstract

Advances in autonomy have the potential to reshape the landscape of the modern world. Yet, research on human-machine interaction is needed to better understand the dynamic exchanges required between humans and machines in order to optimize human reliance on novel technologies. A key aspect of that exchange involves the notion of transparency as humans and machines require shared awareness and shared intent for optimal team work. Questions remain however, regarding how to represent information in order to generate shared awareness and intent in a human-machine context. The current paper will review a recent model of human-robot transparency and will propose a number of methods to foster transparency between humans and machines.

Determinants of system transparency and its influence on trust in and reliance on unmanned robotic systems

“…In order for the agent to be optimally transparent to the human operator, its Beliefs-Desires-Intentions need to be effectively conveyed to the human when appropriate. Conveying the agent's BDI to the human operator should, in theory, benefit the human-agent team's shared mental model and therefore human-agent teaming, although empirical evidence is still lacking to establish that connection [35,39].…”

“…Chen et al propose a model of agent transparency to support operator situation awareness (SA) of the mission environment involving the agent -the SA-based Agent Transparency (SAT; Figure 1) -based on the theory of SA [28], the BDI (Beliefs, Desires, Intentions) Agent Framework [29], Agent's 3P's (purpose, process, and performance --described below) [30], and other relevant previous work [27,[31][32][33][34][35]. According to Endsley [28], SA is "a state of knowledge" of "the state of a dynamic environment" (p. 36).…”

Transparency and Explanation in Deep Reinforcement Learning Neural Networks

“…Autonomous agents have been increasingly used in multiple applications ranging from search and rescue, civilian and military emergency response, home and work environment services, transportation and many others. As these agents become more sophisticated and independent via learning and interaction, it is critical for their human counterparts to understand their behaviors, the reasoning process behind those behaviors, and the expected outcomes to properly calibrate their trust in the systems and make appropriate decisions (de Visser et al 2014) (Lee and See 2004) (Mercado et al 2016). Indeed, past studies have shown that humans sometimes question the accuracy and effectiveness of agents actions due to the human's difficulties understanding the state/status of the agent (Bitan and Meyer 2007) (Seppelt and Lee 2007) (Stanton, Young, and Walker 2007) and the rationales behind the behaviors (Linegang et al 2006).…”

“…All rights reserved. ) (Lyons and Havig 2014), we use, with minor variation, the definition proposed by Chen and colleagues : "Agent transparency is the quality of an interface (e.g. visual, linguistic) pertaining to its abilities to afford an operators comprehension about an intelligent agent's intent, performance, future plans, and reasoning process".…”

“…More specifically, recent work has found outstanding performances of deep reinforcement learning (DRL) models on Atari 2600 games, by using only raw pixels to make decisions. (Mnih et al 2015). However, DNNs are extremely opaque i.e., they cannot produce human understandable accounts of their reasoning processes or explanations.…”