When to (or not to) trust intelligent machines: Insights from an evolutionary game theory analysis of trust in repeated games

Abstract

The actions of intelligent agents, such as chatbots, recommender systems, and virtual assistants are typically not fully transparent to the user. Consequently, using such an agent involves the user exposing themselves to the risk that the agent may act in a way opposed to the user's goals. It is often argued that people use trust as a cognitive shortcut to reduce the complexity of such interactions. Here we formalise this by using the methods of evolutionary game theory to study the viability of trust-based strategies in repeated games. These are reciprocal strategies that cooperate as long as the other player is observed to be cooperating. Unlike classic reciprocal strategies, once mutual cooperation has been observed for a threshold number of rounds they stop checking their co-player's behaviour every round, and instead only check with some probability. By doing so, they reduce the opportunity cost of verifying whether the action of their co-player was actually cooperative. We demonstrate that these trust-based strategies can outcompete strategies that are always conditional, such as Tit-for-Tat, when the opportunity cost is non-negligible. We argue that this cost is likely to be greater when the interaction is between people and intelligent agents, because of the reduced transparency of the agent. Consequently, we expect people to use trust-based strategies more frequently in

Misperception influence on zero-determinant strategies in iterated Prisoner’s Dilemma

“…Consider an IPD game with misperception such as implementation errors and observation errors 22 , 23 , 31 . Due to the misperception, the parameter in the real game changes from to , and only player X notices the change.…”

Early exclusion leads to cyclical cooperation in repeated group interactions

“…Inspired by the above-mentioned behavioural experiments, our mathematical model considers that excluders need to pay a permanent cost of monitoring to identify other players' behaviours during the repeated group interaction process. Indeed, such monitoring cost also called opportunity cost in the previous work [65], has been used to describe the transparency between individuals in game interactions [66]. This realistic set-up makes our model more reasonable and brings a step closer towards understanding how social exclusion strategy works in the realistic case.…”

A Regulation Dilemma in Artificial Intelligence Development

“…We have described how evolutionary modelling and simulations using game theory can be powerful to generate useful insights into the behavioural dynamics and the impact of interventions, in the context of AI development and governance. This approach has been widely adopted to study biological and artificial life systems (Nowak, 2006;Andras et al, 2018;Perc et al, 2017;Smaldino and Lubell, 2014;Han et al, 2021c), which once again has here shown its usefulness to study a complex issue of significant importance.…”