Human-Autonomy Teaming and Agent Transparency

Chen, Jessie Y. C.; Selkowitz, Anthony R.; Stowers, Kimberly; Lakhmani, Shan; Barnes, Michael

doi:10.1145/3029798.3038339

Cited by 8 publications

(5 citation statements)

References 9 publications

(10 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Empirical research demonstrates the positive impact of AI transparency and explainability on trust [e.g. 48,49,50]. Experimental research undertaken in military settings indicates that when human operators and AI agents collaborate, increased transparency enhances trust [48,49].…”

Section: Transparency and Explainabilitymentioning

confidence: 99%

See 1 more Smart Citation

A Review of Trust in Artificial Intelligence: Challenges, Vulnerabilities and Future Directions

Lockey

Gillespie

Holm³

et al. 2021

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

Artificial Intelligence (AI) can benefit society, but it is also fraught with risks. Societal adoption of AI is recognized to depend on stakeholder trust in AI, yet the literature on trust in AI is fragmented, and little is known about the vulnerabilities faced by different stakeholders, making it is difficult to draw on this evidence-base to inform practice and policy. We undertake a literature review to take stock of what is known about the antecedents of trust in AI, and organize our findings around five trust challenges unique to or exacerbated by AI. Further, we develop a concept matrix identifying the key vulnerabilities to stakeholders raised by each of the challenges, and propose a multi-stakeholder approach to future research.

show abstract

Section: Transparency and Explainabilitymentioning

confidence: 99%

“…48,49,50]. Experimental research undertaken in military settings indicates that when human operators and AI agents collaborate, increased transparency enhances trust [48,49]. Explanations have been shown to increase trust in the results of a product release planning tool [51].…”

Section: Transparency and Explainabilitymentioning

confidence: 99%

A Review of Trust in Artificial Intelligence: Challenges, Vulnerabilities and Future Directions

Lockey

Gillespie

Holm³

et al. 2021

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

show abstract

“…Transparency has, in telerobotic systems and human-robot teams, reduced operator workload, facilitated operator comprehension, mitigated errors (Breazeal, Kidd, Thomaz, Hoffman, & Berlin, 2005), improved usability (Berman & Ganel, 2018), and positively impacted perceived system dependability (Vitale et al, 2018). Identifying and affirming which interface symbology supports developing appropriate mental models can improve overall usability of human-robot interaction design (J. Y. C. Chen, Selkowitz, Stowers, Lakhmani, & Barnes, 2016). Robots that increased their transparency levels improved human operator performance, but the perceived usability did not improve (Stowers et al, 2016).…”

Section: Usabilitymentioning

confidence: 99%

“…Presenting the system’s uncertainty information may have introduced ambiguity, lack of relevance, and incomplete knowledge of the system’s operational capabilities, causing mixed human operator responses (Helldin, 2014). The Intelligent Multi-UxV Planner with Adaptive Collaborative/Control Technologies was used to evaluate the impact of transparency on performance, trust (J. Y. C. Chen, Selkowitz, Stowers, Lakhmani, & Barnes, 2017), workload (Mercado et al, 2016), situation awareness, and reliance (Stowers et al, 2016).…”

Section: Transparency Factorsmentioning

confidence: 99%

Transparency: Transitioning From Human–Machine Systems to Human-Swarm Systems

Roundtree

Goodrich

Adams

2019

Journal of Cognitive Engineering and Decision Making

View full text Add to dashboard Cite

Swarm robotic systems are gaining in interest with the prospect of their use for various applications, including monitoring, tracking, infrastructure support, and protection. Prior human-swarm system research investigated transparency for these systems, but assumed perfect communication scenarios. Real-world human-swarm systems will not have perfect communication due to human processing limitations caused by a large number of individual entities, emergent behavior due to varying environments, and bandwidth issues. Factors that affect transparency or are influenced by transparency were identified from various human–machine system domains; however, understanding the relationship between factors can help designers identify what criteria can be implemented to achieve transparency for swarms. Challenges that may arise based on transparency criteria from human–machine systems are examined to identify improvements for spatial swarm systems.

show abstract

“…Human-autonomy teaming (HAT) is a term used to describe humans and intelligent, autonomous agents working interdependently toward a common goal [1,2]. McNeese specifies HAT as at least one human working cooperatively with at least one autonomous agent [3].…”

Section: Introductionmentioning

confidence: 99%

Onto4MAT: A Swarm Shepherding Ontology for Generalised Multi-Agent Teaming

Hepworth¹,

Baxter²,

Abbass³

2022

Preprint

View full text Add to dashboard Cite

Research in multi-agent teaming has increased substantially over recent years, with knowledge-based systems to support teaming processes typically focused on delivering functional (communicative) solutions for a team to act meaningfully in response to direction. Enabling humans to effectively interact and team with a swarm of autonomous cognitive agents is an open research challenge in Human-Swarm Teaming research, partially due to the focus on developing the enabling architectures to support these systems. Typically, bi-directional transparency and shared semantic understanding between agents has not prioritised a designed mechanism in Human-Swarm Teaming, potentially limiting how a human and a swarm team can share understanding and information-data through concepts and contexts-to achieve a goal. To address this, we provide a formal knowledge representation design that enables the swarm Artificial Intelligence to reason about its environment and system, ultimately achieving a shared goal. We propose the Ontology for Generalised Multi-Agent Teaming, Onto4MAT, to enable more effective teaming between humans and teams through the biologically-inspired approach of shepherding.

show abstract

Human-Autonomy Teaming and Agent Transparency

Cited by 8 publications

References 9 publications

A Review of Trust in Artificial Intelligence: Challenges, Vulnerabilities and Future Directions

A Review of Trust in Artificial Intelligence: Challenges, Vulnerabilities and Future Directions

Transparency: Transitioning From Human–Machine Systems to Human-Swarm Systems

Onto4MAT: A Swarm Shepherding Ontology for Generalised Multi-Agent Teaming

Contact Info

Product

Resources

About