Approaches for assessing communication in human-autonomy teams

Abstract

Evaluation of team communication can provide critical insights into team dynamics, cohesion, trust, and performance on joint tasks. Although many communication-based measures have been tested and validated for human teams, this review article extends this research by identifying key approaches specific to human-autonomy teams. It is not possible to identify all approaches for all situations, though the following seem to generalize and support multi-size teams and a variety of military operations. Therefore, this article will outline several key approaches to assessing communication, associated data requirements, example applications, verification of methods through HAT use cases, and lessons learned, where applicable. Some approaches are based on the structure of team communication; others draw from dynamical systems theory to consider perspectives across different timescales; other approaches leverage features of team members’ voices or facial expressions to detect emotional states that can provide windows into other workings of the team; still others consider the content of communication to produce insights. Taken together, these approaches comprise a varied toolkit for deriving critical information about how team interactions affect, and are affected by, coordination, trust, cohesion, and performance outcomes. Future research directions describe four critical areas for further study of communication in human-autonomy teams.

Communication Metrics for Human-Autonomy Teaming: Lessons Learned from us Army Gunnery Field Experiments

“…Whenever possible, multiple measures should be used that consider different aspects and dimensions of team communication, to more fully leverage the richness of communication as a teamwork behavior and vehicle for team cognition. The approaches we utilized for this data set should not be considered exhaustive, however; for a review of additional communication measures aimed at HAT, see Baker et al (2021).…”

“…Poor communication can lead to poor teamwork; failures in communication have long been identified as key factors in naval groundings and aviation accidents (Macrae 2009;Sexton and Helmreich 2000). Measures of communication can be implemented to assess the health and functioning of human-only teams, but the added complexities of autonomous, intelligent systems can so broadly rewrite team dynamics that our understanding of the relationships between these metrics and team performance must be re-evaluated to understand whether they can be generalized from the humanonly context to the HAT context (Baker et al, 2021).…”

Human-Agent Teaming: A System-Theoretic Overview

“…HAI also considers social interaction between humans and robots [38], trust [39], [40], and acceptance [41], [42]. Various user studies and experiments are conducted in HAI to evaluate human-autonomy systems' performance and identify areas of improvement [43], [44]. The HAI field can also focuses on developing agents with humanlike behavior and intelligence using machine learning, AI, and cognitive architectures [45], [46].…”

Investigating Cyber Attacker Team Cognition

“…Several studies use content analysis to categorize words or messages according to meaning (e.g., "request," "update"). Some types of content analyses are very labor-intensive, although some automated analysis approaches do exist (Baker et al, 2021). It is more likely that quick research benefits will come from studying communications flow because it is easier to collect and analyze by avoiding content analysis.…”