Deep learning control of artificial avatars in group coordination tasks

Abstract

In many joint-action scenarios, humans and robots have to coordinate their movements to accomplish a given shared task. Lifting an object together, sawing a wood log, transferring objects from a point to another are all examples where motor coordination between humans and machines is a crucial requirement. While the dyadic coordination between a human and a robot has been studied in previous investigations, the multi-agent scenario in which a robot has to be integrated into a human group still remains a less explored field of research. In this paper we discuss how to synthesise an artificial agent able to coordinate its motion in human ensembles. Driven by a control architecture based on deep reinforcement learning, such an artificial agent will be able to autonomously move itself in order to synchronise its motion with that of the group while exhibiting human-like kinematic features. As a paradigmatic coordination task we take a group version of the so-called mirrorgame which is highlighted as a good benchmark in the human movement literature.

Dynamic Input Deep Learning Control of Artificial Avatars in a Multi-Agent Joint Motor Task

“…Using the deep Q-network (DQN) learning algorithm ( Mnih et al, 2015 ), the cyberplayer in Lombardi et al (2019) was synthesised as an artificial agent able to train itself by observing a specific target player (TP) in order to extract his/her kinematic motor characteristics from the data.…”

“…A crucial problem when introducing an artificial avatar, or robot, in the group playing the game [as for instance done in Zhai et al (2016) , Zhai et al (2017) ] is to design a control architecture to make the avatar observe the motion of the other group members and coordinate its motion with them in a natural “human-like” way ( Lombardi et al, 2019 ; Lombardi et al, 2021 ). In this paper we overcome some of the existing limitations on scalability and flexibility of previous proposed designs ( Lombardi et al, 2019 ) by developing an alternative strategy based on deep reinforcement learning. Specifically, our control framework allows the cyberplayer (CP) to perform the task with the others while, at the same time, exhibiting human-like kinematic features.…”

“…A preliminary approach to solve the problem was presented in our previous work ( Lombardi et al, 2019 ). In Lombardi et al (2019) , we adopted a different learning approach where the learning agent plays against an “average” player in what boils down to a dyadic interaction between the agent and an average of his neighbours.…”

“…A preliminary approach to solve the problem was presented in our previous work ( Lombardi et al, 2019 ). In Lombardi et al (2019) , we adopted a different learning approach where the learning agent plays against an “average” player in what boils down to a dyadic interaction between the agent and an average of his neighbours. Therefore, Lombardi et al (2019) can be seen as an intermediate step between the dyadic case proposed in our earlier works and the multi-agent case investigated in the current manuscript.…”

“…In Lombardi et al (2019) , we adopted a different learning approach where the learning agent plays against an “average” player in what boils down to a dyadic interaction between the agent and an average of his neighbours. Therefore, Lombardi et al (2019) can be seen as an intermediate step between the dyadic case proposed in our earlier works and the multi-agent case investigated in the current manuscript. However, the main drawback of this approach is the assumption that the other players in the group adjust their motion on a real time average of the positions of their neighbours.…”