Using a Stochastic Agent Model to Optimize Performance in Divergent Interest Tacit Coordination Games

Abstract

In recent years collaborative robots have become major market drivers in industry 5.0, which aims to incorporate them alongside humans in a wide array of settings ranging from welding to rehabilitation. Improving human–machine collaboration entails using computational algorithms that will save processing as well as communication cost. In this study we have constructed an agent that can choose when to cooperate using an optimal strategy. The agent was designed to operate in the context of divergent interest tacit coordination games in which communication between the players is not possible and the payoff is not symmetric. The agent’s model was based on a behavioral model that can predict the probability of a player converging on prominent solutions with salient features (e.g., focal points) based on the player’s Social Value Orientation (SVO) and the specific game features. The SVO theory pertains to the preferences of decision makers when allocating joint resources between themselves and another player in the context of behavioral game theory. The agent selected stochastically between one of two possible policies, a greedy or a cooperative policy, based on the probability of a player to converge on a focal point. The distribution of the number of points obtained by the autonomous agent incorporating the SVO in the model was better than the results obtained by the human players who played against each other (i.e., the distribution associated with the agent had a higher mean value). Moreover, the distribution of points gained by the agent was better than any of the separate strategies the agent could choose from, namely, always choosing a greedy or a focal point solution. To the best of our knowledge, this is the first attempt to construct an intelligent agent that maximizes its utility by incorporating the belief system of the player in the context of tacit bargaining. This reward-maximizing strategy selection process based on the SVO can also be potentially applied in other human–machine contexts, including multiagent systems.

An Electrophysiological Model for Assessing Cognitive Load in Tacit Coordination Games

“…This finding is compatible with the level-k theory, which defines picking as level k = 0 and coordination as level k > 0 [59][60][61][62]. In previous studies, we have shown that considerable variability exits in behavioral coordination ability exists between players [23][24][25]30,[63][64][65][66]. In the current study, we have shown that as coordination ability increases the TBR which reflects cognitive load, decreases.…”

Level-K Classification from EEG Signals Using Transfer Learning

“…To that end, in future studies, it is recommended to utilize inverse-problem techniques such as LORETA [ 55 , 56 ]. Finally, behavioral and electrophysiological data of human agents (e.g., [ 2 , 9 , 57 , 58 , 59 ]) gained from these studies might aid in constructing brain–computer interfaces as well as autonomous agents. In this study, we used transfer learning when the training set for the embedding network was a general set of images (ImageNet) that did not include EEG signals.…”

“…This will show that the level-k model can also be validated by electrophysiological correlates and not only by behavioral indices. This validation may potentially enable the construction of more accurate models for human–agent interactions [ 9 ]. To that end, we have first used methods of feature extraction and classification based on conventional machine learning techniques, such as computing the relative energy in each frequency band and applying standard predictive models such as random forest (see Appendix D ).…”

Modeling and predicting individual tacit coordination ability

“…This multimodal distribution might corroborate cognitive hierarchy theory [ 36 – 38 ], which postulates that individuals differ in their depth of reasoning. By this account, an agent is bounded by the k steps of reasoning they can perform [ 39 , 40 ]. Thus, each of the three Gaussians found in the current study may correspond to a different level k that bounds the best response given by the players.…”

“…The focal points that were modeled in our study are based on spatial properties (e.g., closeness, equality, and accession). Consequently, we expect the model we have constructed to be applicable to other contexts where focal points are based on spatial cues, e.g., “Bargaining Table” [ 14 , 35 , 40 , 48 ] and “Moving Discs” [ 18 ]. However, in other cases, where focal points are based on non-spatial features (e.g., the “word Selection” task based on semantic meaning [ 18 , 24 , 42 , 49 ]) our model should be modified accordingly.…”