The Ethical and Legal Case Against Autonomy in Weapons Systems

Abstract

In order to be counted as autonomous, a weapons system must perform the critical functions of target selection and engagement without any intervention by human operators. Human rights organizations, as well as a growing number of States, have been arguing for banning weapons systems satisfying this condition – that are usually referred to as autonomous weapons system (AWS) in this account – and for maintaining meaningful human control (MHC) over any weapons systems. This twofold goal has been pursued by leveraging on ethical and legal arguments, which spell out a variety of deontological or consequentialist reasons. Roughly speaking, deontological arguments support the conclusion that by deploying AWS one is likely or even bound to violate moral and legal obligations of special sorts of agents (military commanders and operators) or moral and legal rights of special sorts of patients (AWS potential victims). Consequentialist arguments substantiate the conclusion that prohibiting AWS is expected to protect peace and security, thereby enhancing collective human welfare, more effectively than the incompatible choice of permitting their use. Contrary to a widespread view, this paper argues that deontological and consequentialist reasons can be coherently combined so as to provide mutually reinforcing ethical and legal reasons for banning AWS. To this end, a confluence model is set forth that enables one to solve potential conflicts between these two approaches by prioritizing deontological arguments over consequentialist ones. Finally, it is maintained that the proposed confluence model significantly bears on the issue of what it is to exercise genuine MHC on existing and future AWS. Indeed, full autonomy is allowed by the confluence model in the case of some anti-materiel defensive AWS; it is to be curbed instead in the case of both lethal AWS and future AWS which may seriously jeopardize peace and stability.

Autonomy in surgical robots and its meaningful human control

“…Discussion of what MHC should amount to in the case of AWS is well under way in both academic [62][63][64] and international policy fora [65]. Moreover, applications of the MHC concept to increasingly autonomous driverless vehicles and other autonomous robotic and computational systems are being actively explored [6].…”

Autonomous weapons systems, killer robots and human dignity

“…The structure of law and the processes of justice require the presence of a human as a legal agent, and the case against AWS is both a legal and a moral one. Amoroso and Tamburrini (2017) distinguish two kinds of argument made about AWS: deontological and consequentialist. They identify three main deontological arguments: (a) that AWS would be unable to conform to IHL and IHRL rules governing the use of lethal force; (b) that AWS would create an accountability gap; and (c) that deployment of AWS would be contrary to human dignity and the requirement that 'the taking of human life should be reserved to human decision-makers'.…”

“…Tamburrini (2016) argues that swarms of AWS could weaken traditional nuclear deterrent factors based on mutually assured destruction (swarms of AWS could be used to deliver destructive attacks on strategic nuclear and eliminate an opponent's second strike nuclear capability, increasing preference for first strike strategies). Amoroso and Tamburrini (2017) point out that AWS even without the 'lethal' element, if used to destroy buildings or infrastructure, would still have a global destabilising effect.…”

“…On the basis of the preceding account, three main categories of argument against AWS are identified here: In contrast to Amoroso and Tamburrini (2017), arguments based on the abilities of AWS (i), are distinguished here from arguments about the need for human judgement (ii). This is in order to place an increased focus on what weapons systems and artificial intelligence are capable of, and on the differences between humans and computational artifacts or machines.…”

“…There is also the possibility of combining the arguments, instead of selecting the most appropriate. Amoroso and Tamburrini (2017) present an interesting idea for combining deontological and consequential reasons by means of a confluence model that resolves potential conflicts between deontological and consequential arguments by prioritising deontological arguments.…”