A Reputation Game Simulation: Emergent Social Phenomena from Information Theory

Abstract

Reputation is a central element of social communications, be it with human or artificial intelligence (AI), and as such can be the primary target of malicious communication strategies. There is already a vast amount of literature on trust networks and their dynamics using Bayesian principles and involving Theory of Mind models. An issue for these simulations can be the amount of information that can be stored and managed using discretizing variables and hard thresholds. Here a novel approach to the way information is updated that accounts for knowledge uncertainty and is closer to reality is proposed. Agents use information compression techniques to capture their complex environment and store it in their finite memories. The loss of information that results from this leads to emergent phenomena, such as echo chambers, self-deception, deception symbiosis, and freezing of group opinions. Various malicious strategies of agents are studied for their impact on group sociology, like sycophancy, egocentricity, pathological lying, and aggressiveness. Our set-up already provides insights into social interactions and can be used to investigate the effects of various communication strategies and find ways to counteract malicious ones. Eventually this work should help to safeguard the design of non-abusive AI systems.

Information and Agreement in the Reputation Game Simulation

“…In previous works [ 1 , 20 ] we used the optimal belief compression as described in Section 2.5.3 , as it follows information theoretical principles and ensures minimal information loss [ 21 ]. However, there are two reasons why we now question this choice.…”

“…In a frozen climate, for example, both and would be rather small, whereas, in a toxic social climate full of chaos, both can increase by orders of magnitude. More details on the lie construction, and especially on the determination of and , can be found in [ 1 ].…”

“…As already explained in Section 2.2 , agents become skeptical if the received message deviates too much from their prior belief. For further details on how to calculate the likelihood ratio from these different criteria, see [ 1 ].…”

“…This prevents the agents from taking the same opinion they hear more than once as completely new each time, but filtering out already known information. How is calculated in detail can again be found in [ 1 ].…”

“…By simulating communication dynamics only based on the assumption that agents do not necessarily communicate truthfully all the time we will replicate phenomena that are known in real-life sociology. We have already done so in previous works on reputation game simulations [ 1 ], where several types of agents with different strategies were investigated. However, in this work we will only focus on one kind of agent, called ordinary agents, but use significantly larger system sizes of 50 agents.…”