Virtual Reality for Immersive Human Machine Teaming with Vehicles

A Preliminary Study on Virtual Reality Tools in Human-Robot Interaction

“…Information is the most frequently adopted speech act, allowing to notify a teammate about: an intention [23,1], the current activity, the measured dis-tance from the target [23] or from the human [36], object properties [37] or trajectory [2]. Human presence (awareness of the person in the environment) is another relevant element considered in different studies [31], together with information about position with respect to an object [10,22] or a teammate [32], current status [36] and achieved performance [36,26].…”

“…An unidirectional speech act category, expressing the commands given by the human to the robot, is represented by the command speech act class, in which two sub-classes have been identified: motion, in which the android must reach a new position [23,22], change its velocity, or stop its operation [23,7], and action, that consists of performing specific activities, such as following a person [16,32], picking up [7] or positioning an object [17].…”

“…The first modality is exploited, for example, by a social robot tour guide in the virtual museum, with the aim of showing and talking about existing artworks of the Metropolitan Museum of Art of New York [37]. Moreover, vocal communication allows the exchange of mission details between humans and robots in marine [32] and military [7] scenarios; to update the teammate about processes, so to increase awareness of system's features and to improve performances, by clarifying ambiguous situations, giving suggestions to correctly complete a task [34] and communicating the final outcome [26].…”

“…Pointing is not only employed to provide a specific position to reach, but also to indicate an object, as presented in [16]. The authors of this article stress the importance of such interaction channel in commanding a robot to follow someone, by only pointing the person [32].…”

Remote research methods for Human–AI–Robot Teaming

“…By contrast to these primarily virtual task environments, The Project Aquaticus testbed's task environment features teams of multiple humans and autonomous robots competing in a game of capture the flag using selfpropelled kayaks (Novitzky et al, 2016). This testbed has recently adapted to function in a lab setting with an immersive virtual task environment through the use of virtual reality headsets and other lab technology (Novitzky et al, 2020). Other HART testbeds have been designed with remotely distributed data collection methods and task environments.…”