Meaningful human control: actionable properties for AI system development

Abstract

How can humans remain in control of artificial intelligence (AI)-based systems designed to perform tasks autonomously? Such systems are increasingly ubiquitous, creating benefits - but also undesirable situations where moral responsibility for their actions cannot be properly attributed to any particular person or group. The concept of meaningful human control has been proposed to address responsibility gaps and mitigate them by establishing conditions that enable a proper attribution of responsibility for humans; however, clear requirements for researchers, designers, and engineers are yet inexistent, making the development of AI-based systems that remain under meaningful human control challenging. In this paper, we address the gap between philosophical theory and engineering practice by identifying, through an iterative process of abductive thinking, four actionable properties for AI-based systems under meaningful human control, which we discuss making use of two applications scenarios: automated vehicles and AI-based hiring. First, a system in which humans and AI algorithms interact should have an explicitly defined domain of morally loaded situations within which the system ought to operate. Second, humans and AI agents within the system should have appropriate and mutually compatible representations. Third, responsibility attributed to a human should be commensurate with that human’s ability and authority to control the system. Fourth, there should be explicit links between the actions of the AI agents and actions of humans who are aware of their moral responsibility. We argue that these four properties will support practically minded professionals to take concrete steps toward designing and engineering for AI systems that facilitate meaningful human control.

Meaningful human control and variable autonomy in human-robot teams for firefighting

“…Ultimately, the goal should be to get the highest possible traceability score rather than a minimum sufficient value. This is in line with some earlier interpretations of meaningful human control as ratio rather than binary ( Calvert et al, 2020 ; Cavalcante Siebert et al, 2022 ). On the other hand, it contradicts the discussion on defining how much of each of the four properties for human-robot teams to be under meaningful human control is sufficient ( Cavalcante Siebert et al, 2022 ).…”

“…Another operationalization are the four necessary properties for human-robot teams to be under meaningful human control ( Cavalcante Siebert et al, 2022 ). The first property requires an explicitly specified moral operational design domain where the robot should adhere to.…”

Meaningful Human Control over AI for Health? A Review

“…Recent publications address this difficulty and propose further cornerstones to help implement the concept from theoretical considerations into practice. 33 The discussion about MHC in the health sector, which started in 2016, is still in its infancy. The high number of unspecified codes in the individual code groups within our results shows that the concept is sometimes only treated superficially.…”

“…As Luciano Cavalcante Siebert and colleagues note: 'Meaningful human control refers not only to the development of the AI agent, but also to the design of the sociotechnical environment that surrounds it, including social and institutional practices'. 33 MHC is applied in specific, often pre-existing situations (eg, surgery or medical diagnosis) that shape the concept of MHC. Since MHC is influenced and shaped by this particular sociotechnical environment, the first question is which institutions and actors need to work together to make MHC possible in different health settings in the first place.…”

Grasping AI: experiential exercises for designers

“…Designers have their own strategies for making use of, critiquing and appropriating new technologies (Westerlund and Wetter-Edman 2017), so there is an interest in understanding what designerly methods could reveal about human-AI relations, particularly where it involves interactions between humans and technological systems-considering the "social, political, ethical, cultural, and environmental factors of implementing AI into daily human-to-computer interactions" (Wong 2018). Design research methods, speculations (Auger 2013;Kirman et al 2022), fictioning (Forlano and Mathew 2014;Wong et al 2017;Troiano et al 2021;Benjamin et al 2023), probes and toolkits (Sanders and Stappers 2014), more than human design (Coulton and Lindley 2019) and the general practices of Research through Design (RtD) (Giaccardi 2019; Stappers and Giaccardi 2017), are all well suited to thinking into the socio-technical aspects (Holton and Boyd 2021; Sartori and Theodorou 2022;Theodorou and Dignum 2020), possibilities, and implication of AI in everyday life, just as they have been applied to understanding digital sensing technologies (Pierce 2021), blockchains (Murray-Rust et al 2022a), the future of automation (Cavalcante Siebert et al 2022) and so on.…”