Mediating artificial intelligence developments through negative and positive incentives

Han, The Anh; Lenaerts, Tom; Santos, Francisco C.

doi:10.1371/journal.pone.0244592

Cited by 28 publications

(34 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The DSAIR model and associated analysis provides thus an instrument for researchers interested in AI regulation and policy making to think about the supporting mechanisms (such as suitable rewards and sanctions) (Sigmund, 2010;Sotala & Yampolskiy, 2014;Szolnoki & Perc, 2013;Han, Pereira, & Lenaerts, 2015, 2019Vinuesa et al, 2020) needed to mediate a given race; for preliminary results, see our recent work in (Han et al, 2020). In the early DSAI, controlling the development speed of AI teams appears essential.…”

Section: Discussionmentioning

confidence: 99%

To Regulate or Not: A Social Dynamics Analysis of an Idealised AI Race

Santos¹,

Lenaerts

2020

jair

Self Cite

View full text Add to dashboard Cite

Rapid technological advancements in Artificial Intelligence (AI), as well as the growing deployment of intelligent technologies in new application domains, have generated serious anxiety and a fear of missing out among different stake-holders, fostering a racing narrative. Whether real or not, the belief in such a race for domain supremacy through AI, can make it real simply from its consequences, as put forward by the Thomas theorem. These consequences may be negative, as racing for technological supremacy creates a complex ecology of choices that could push stake-holders to underestimate or even ignore ethical and safety procedures. As a consequence, different actors are urging to consider both the normative and social impact of these technological advancements, contemplating the use of the precautionary principle in AI innovation and research. Yet, given the breadth and depth of AI and its advances, it is difficult to assess which technology needs regulation and when. As there is no easy access to data describing this alleged AI race, theoretical models are necessary to understand its potential dynamics, allowing for the identification of when procedures need to be put in place to favour outcomes beneficial for all. We show that, next to the risks of setbacks and being reprimanded for unsafe behaviour, the time-scale in which domain supremacy can be achieved plays a crucial role. When this can be achieved in a short term, those who completely ignore the safety precautions are bound to win the race but at a cost to society, apparently requiring regulatory actions. Our analysis reveals that imposing regulations for all risk and timing conditions may not have the anticipated effect as only for specific conditions a dilemma arises between what is individually preferred and globally beneficial. Similar observations can be made for the long-term development case. Yet different from the short-term situation, conditions can be identified that require the promotion of risk-taking as opposed to compliance with safety regulations in order to improve social welfare. These results remain robust both when two or several actors are involved in the race and when collective rather than individual setbacks are produced by risk-taking behaviour. When defining codes of conduct and regulatory policies for applications of AI, a clear understanding of the time-scale of the race is thus required, as this may induce important non-trivial effects. This article is part of the special track on AI and Society.

show abstract

Section: Discussionmentioning

confidence: 99%

To Regulate or Not: A Social Dynamics Analysis of an Idealised AI Race

Santos¹,

Lenaerts

2020

jair

Self Cite

View full text Add to dashboard Cite

show abstract

“…Indeed, we explicitly tested in our simulations what would happen if always companies that take risks are sanctioned (Han et al, 2021b), reducing their speed but at the cost of speed reduction by the sanctioning party. As anticipated, over-regulation, conducive to beneficial innovation being stifled, occurred whenever the gain from speeding up outbenefited the taking of risk.…”

Section: Lessons For Ai Governance Policiesmentioning

confidence: 99%

“…Here, we summarise our previous works (Han et al, 2020(Han et al, , 2021b examining this problem theoretically, resorting to a novel innovation dilemma where technologists can choose a safe (SAFE) vs risk-taking (UNSAFE) course of development. Companies race towards the deployment of some AI -based product in some domain X.…”

Section: Introductionmentioning

confidence: 99%

“…We next explored ways to influence it towards safe and beneficial outcomes, namely when and how to sanction unsafe decisions made by stake-holders or reward compliant ones. Finally, we identify when regulations need to be put in place to favour outcomes most beneficial for all, but at the same time taking care to avoid strict regulations that would be introduced too early and thereby stifle innovation (Han et al, 2020(Han et al, , 2021b(Han et al, , 2019.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Regulation Dilemma in Artificial Intelligence Development

Santos

Lenaerts

2021

The 2021 Conference on Artificial Life

Self Cite

View full text Add to dashboard Cite

We examine a social dilemma that arises with the advancement of technologies such as AI, where technologists can choose a safe (SAFE) vs risk-taking (UNSAFE) course of development. SAFE is costlier and takes more time to implement than UNSAFE, allowing UNSAFE strategists to further claim significant benefits from reaching supremacy in a certain technology. Collectively, SAFE is the preferred choice when the risk is sufficiently high, while risk-taking is preferred otherwise. Given the advantage of risk-taking behaviour in terms of cost and speed, a social dilemma arises when the risk is not high enough to make SAFE the preferred individual choice, enabling UNSAFE to prevail when it is not collectively preferred (leading to a smaller population/social welfare). We show that the range of risk probabilities where the social dilemma arises depends on many factors, the most important among them are the time-scale to reach supremacy in a given domain (i.e. short-term vs long-term AI) and the speed gain by ignoring safety measures. Moreover, given the more complex nature of this scenario, we show that incentives such as reward and punishment (for example, for the purpose of technology regulation) are much more challenging to supply correctly than in case of cooperation dilemmas such as the Prisoner's Dilemma and the Public Good Games.

show abstract

“…Institutional enforcement mechanisms are crucial for enabling large-scale cooperation. Most modern societies implemented different forms of institutions for governing and promoting collective behaviors, including cooperation, coordination, and technology innovation [Ostrom, 1990, Bowles, 2009, Bowles and Gintis, 2002, Bardhan, 2005, Han et al, 2021, Scotchmer, 2004].…”

Section: Introductionmentioning

confidence: 99%

Cost efficiency of institutional incentives in finite populations

Duong

Han

2021

Preprint

View full text Add to dashboard Cite

Institutions can provide incentives to increase cooperation behaviour in a population where this behaviour is infrequent. This process is costly, and it is thus important to optimize the overall spending. This problem can be mathematically formulated as a multi-objective optimization problem where one wishes to minimize the cost of providing incentives while ensuring a desired level of cooperation within the population. In this paper, we provide a rigorous analysis for this problem. We study cooperation dilemmas in both the pairwise (the Donation game) and multi-player (the Public Goods game) settings. These are well-known models for studying cooperation since individually defection is always preferred over cooperation while mutual cooperation is the preferred collective outcome for the population as a whole. We prove the regularity of the (total incentive) cost function, characterize its asymptotic limits (infinite population, weak selection and large selection) and show exactly when reward or punishment is more efficient. We prove that the cost function exhibits a phase transition phenomena when the intensity of selection varies. We calculate the critical threshold in regards to the phase transition and study the optimization problem when the intensity of selection is under and above the critical value. It allows us to provide an exact calculation for the optimal cost of incentive, for a given intensity of selection. Finally, we provide numerical simulations to demonstrate the analytical results. Overall, our analysis provides for the first time a selection-dependent calculation of the optimal cost of institutional incentives (for both reward and punishment) that guarantees a minimum amount of cooperation. It is of crucial importance for real-world applications of institutional incentives since intensity of selection is specific to a given population and the underlying game payoff structure.

show abstract

Mediating artificial intelligence developments through negative and positive incentives

Cited by 28 publications

References 60 publications

To Regulate or Not: A Social Dynamics Analysis of an Idealised AI Race

To Regulate or Not: A Social Dynamics Analysis of an Idealised AI Race

A Regulation Dilemma in Artificial Intelligence Development

Cost efficiency of institutional incentives in finite populations

Contact Info

Product

Resources

About