Telling autonomous systems what to do

Abstract

Recent progress in Artificial Intelligence, sensing and network technology, robotics, and (cloud) computing has enabled the development of intelligent autonomous machine systems. Telling such autonomous systems "what to do" in a responsible way, is a non-trivial task. For intelligent autonomous machines to function in human society and collaborate with humans, we see three challenges ahead affecting meaningful control of autonomous systems. First, autonomous machines are not yet capable of handling failures and unexpected situations. Providing procedures for all possible failures and situations is unfeasible because the state-action space would explode. Machines should therefore become self-aware (self-assessment, self-management) enabling them to handle unexpected situations when they arise. This is a challenge for the computer science community. Second, in order to keep (meaningful) control, humans come into a new role of providing intelligent autonomous machines with objectives or goal functions (including rules, norms, constraints and moral values), specifying the utility of every possible outcome of actions of autonomous machines. Third, in order to be able to collaborate with humans, autonomous systems will require an understanding of (us) humans (i.e., our social, cognitive, affective and physical behaviors) and the ability to engage in partnership interactions (such as explanations of task performances, and the establishment of joint goals and work agreements). These are new challenges for the cognitive ergonomics community. CCS CONCEPTS • Human-centered computing → Interaction design → Interaction design theory, concepts and paradigms;

Hybrid Cognitive-Affective Strategies for AI Safety

“…Among others, self-awareness and proactive AI safety measures have been proposed to increase error tolerance and indirectly support the value specification process. While the focus in this chapter was generally on highly advanced Type I AI, the self-awareness concept has been described as a challenge for the computer science community and been identified as requirement for the meaningful control of any intelligent system in a follow-up work by Werkhoven et al [428] -making self-awareness already essential for present-day projects on intelligent systems. However, for a full account of the value specification subtask, the briefly mentioned concept of a goal function encoding human ethical values and legal constraints is necessary in addition.…”

“…First the attempt of method 3) to try to formulate deontological rules for every situation an intelligent system might encounter in a complex real-world environment is technically impracticable (it leads to a "state-action space explosion" [428]). Conversely, for the utility-based strategy 4), there exist corresponding systems engineering oriented techniques on how to implement run-time adaptive models equipped with a so-called "self-awareness" functionality (self-management, self-assessment and the ability to provide explanations [12]) that would not face such problems.…”

“…In the following, these objective functions that should encode the ethical and legal framework are referred to as ethical goal functions. Third, legal frameworks often leave trade-offs and dilemmas open which the deontological approach cannot directly solve [428], a problem which a utility-based system would not encounter. Fourth, an update of laws in the deontological case will require every manufacturer to costly modify the built-in ethical framework, while the utility-based solution would only require a centralized update of an ethical goal function.…”

“…For this purpose, we establish a set of hybrid and cognitive-affective strategical clusters to foster AI safety -a field which we remodel throughout the thesis and whose problem space we shift and recontextualize. A priori, the necessity to consider "hybrid" solutions emanates from the realization that improvement and enhancement measures for safer AI systems cannot be solely applied to the artificial system in isolation [428]. Instead, AI has to be considered in the wide socio-technological context of its deployment amidst human entities [30].…”

“…Given the breadth of the topic, we paradigmatically take the meaningful control of "intelligent systems" 1 as point of departure -an already practically relevant and graspable contemporary challenge [428] encompassing both control and value alignment issues. After determining AI safety strategies for the meaningful control of intelligent systems from Chapter 2 to 9, we analyze their transferability to other sorts of AI systems in Chapter 10.…”