Toward a Normative Model of Meaningful Human Control over Weapons Systems

Abstract

The notion of meaningful human control (MHC) has gathered overwhelming consensus and interest in the autonomous weapons systems (AWS) debate. By shifting the focus of this debate to MHC, one sidesteps recalcitrant definitional issues about the autonomy of weapons systems and profitably moves the normative discussion forward. Some delegations participating in discussions at the Group of Governmental Experts on Lethal Autonomous Weapons Systems meetings endorsed the notion of MHC with the proviso that one size of human control does not fit all weapons systems and uses thereof. Building on this broad suggestion, we propose a “differentiated”—but also “principled” and “prudential”—framework for MHC over weapons systems. The need for a differentiated approach—namely, an approach acknowledging that the extent of normatively required human control depends on the kind of weapons systems used and contexts of their use—is supported by highlighting major drawbacks of proposed uniform solutions. Within the wide space of differentiated MHC profiles, distinctive ethical and legal reasons are offered for principled solutions that invariably assign to humans the following control roles: (1) “fail-safe actor,” contributing to preventing the weapon's action from resulting in indiscriminate attacks in breach of international humanitarian law; (2) “accountability attractor,” securing legal conditions for international criminal law (ICL) responsibility ascriptions; and (3) “moral agency enactor,” ensuring that decisions affecting the life, physical integrity, and property of people involved in armed conflicts be exclusively taken by moral agents, thereby alleviating the human dignity concerns associated with the autonomous performance of targeting decisions. And the prudential character of our framework is expressed by means of a rule, imposing by default the more stringent levels of human control on weapons targeting. The default rule is motivated by epistemic uncertainties about the behaviors of AWS. Designated exceptions to this rule are admitted only in the framework of an international agreement among states, which expresses the shared conviction that lower levels of human control suffice to preserve the fail-safe actor, accountability attractor, and moral agency enactor requirements on those explicitly listed exceptions. Finally, we maintain that this framework affords an appropriate normative basis for both national arms review policies and binding international regulations on human control of weapons systems.

Single-Pilot Incapacitation in commercial aviation - Evaluation of an Operational Concept

“…However, this tendency is now calling into question the very same principle of human oversight. In fact, a debate is unfolding on whether ensuring control by a responsible human is always feasible/realistic, and in any case whether human control always contributes to promoting the values at stake [14]. As noted above, in fact, SAFELAND seems to open to scenarios where the GSO does not always have final authority over the procedure: in these situations, the human is not able to retain decision making authority, but s/he will rather tend to trust decisions of the machine, except for patently abnormal behaviours or failures.…”

Autonomous Weapon Systems Under International Law

“…Human control consists of predictability and reliability. This refers to the ability to estimate that a certain thing will happen in the future and, the quality of being trustworthy (Amoroso & Tamburrini, 2021).…”

Ética del ecosistema híbrido cognitivo entre el espacio físico y el ciberespacio. Aproximación desde el caso de la inteligencia artificial

“…A partir de esta premisa, la determinación de lo que se denomina "control humano significativo" puede resultar muy compleja y ser diferente en circunstancias variables. Numerosos autores han tratado este tema sin conseguir cerrarlo completamente (Amoroso y Tamburrini, 2019;Cummings, 2019;Merel Ekelhof, 2018; H. Roff y Moyes, 2015; Santoni de Sio y van den Hoven, 2018).…”