Norms Are What Machines Make of Them: Autonomous Weapons Systems and the Normative Implications of Human-Machine Interactions

Abstract

The emergence of autonomous weapons systems (AWS) is increasingly in the academic and public focus. Research largely focuses on the legal and ethical implications of AWS as a new weapons category set to revolutionize the use of force. However, the debate on AWS neglects the question of what introducing these weapons systems could mean for how decisions are made. Pursuing this from a theoretical-conceptual perspective, the article critically analyzes what impact AWS can have on norms as standards of appropriate action. The article draws on the Foucauldian “apparatus of security” to develop a concept that accommodates the role of security technologies for the conceptualization of norms guiding the use of force. It discusses to what extent a technologically mediated construction of a normal reality emerges in the interplay of machinic and human agency and how this leads to the development of norms. The article argues that AWS provide a specific construction of reality in their operation and thereby define procedural norms that tend to replace the deliberative, normative-political decision on when, how, and why to use force. The article is a theoretical-conceptual contribution to the question of why AWS matter and why we should further consider the implications of new arrangements of human-machine interactions in IR.

Practice-based and public-deliberative normativity: retaining human control over the use of force

“…Such assumptions include the level of trust put into system outputs and the practical set-up of human operators as part of the decision-making loop of weapon systems. The repeated, collective performance of a similar range of practices over time ‘makes normal’ and thereby increasingly ‘acceptable’ (Huelss, 2020). Making normal, associated with notions such as ‘the common, the ordinary, the standard, the conventional, the regular’ (Cryle and Stephens, 2017: 1), therefore interrelates with normativity in indicating incremental, collectively shifting understandings of appropriateness.…”

“…Third, by investigating the trajectory of normativity on human control, I make a substantial contribution to the literature on LAWS that has covered four strands of inquiry: (a) the extent to which using LAWS may be compatible with or challenge international law (e.g. Crootof, 2015; Heyns, 2016); (b) their effects on strategic stability and the balance of power (Altmann and Sauer, 2017; Payne, 2021); (c) the ethical consequences of their development (Asaro, 2019; Renic, 2020; Schwarz, 2019); and (d) their potential effect on international norms broadly understood (Bode and Huelss, 2018, 2022; Garcia, 2016; Huelss, 2020; Rosert and Sauer, 2021; Williams, 2021). While some policy studies concentrate on human control (Boulanin et al, 2020), there is no systematic, analytical study examining what shapes normativity in this space.…”

Algorithmic Warfare: Taking Stock of a Research Programme

“…Finally, literature in the field of IR has considered the potential impact of algorithmic warfare on international legal and social norms. Following a broader, constructivist definition of norms as understandings of appropriateness, studies have focused on how practices associated with developing and using weapon systems integrating algorithms in targeting can change what states consider appropriate uses of force Huelss 2021, 2023;Huelss 2020). Scholars have also considered the potential effects of not regulating algorithmic warfare for the humanitarian disarmament regime (Garcia 2016).…”

Taking the trouble: science, technology and security studies

“…This is different from claiming that science and technology are de facto more important today than a few decades ago. Rather it means that recent technological developments like the internet (Carr 2015;Herrera 2002), cybersecurity (Dunn Cavelty 2013; Fichtner 2018) and new weapons (Boyle 2013;Huelss 2019) require that we pay attention to their (in)security politics. However, not all technology-focused research in security studies takes its inspiration from STS, and nor do all studies of security technologies and the intersection of science and (in)security take place within the field of security studies.…”