Augmented Utilitarianism for AGI Safety

Abstract

In the light of ongoing progresses of research on artificial intelligent systems exhibiting a steadily increasing problem-solving ability, the identification of practicable solutions to the value alignment problem in AGI Safety is becoming a matter of urgency. In this context, one preeminent challenge that has been addressed by multiple researchers is the adequate formulation of utility functions or equivalents reliably capturing human ethical conceptions. However, the specification of suitable utility functions harbors the risk of "perverse instantiation" for which no final consensus on responsible proactive countermeasures has been achieved so far. Amidst this background, we propose a novel socio-technological ethical framework denoted Augmented Utilitarianism which directly alleviates the perverse instantiation problem. We elaborate on how augmented by AI and more generally science and technology, it might allow a society to craft and update ethical utility functions while jointly undergoing a dynamical ethical enhancement. Further, we elucidate the need to consider embodied simulations in the design of utility functions for AGIs aligned with human values. Finally, we discuss future prospects regarding the usage of the presented scientifically grounded ethical framework and mention possible challenges.

Hybrid Cognitive-Affective Strategies for AI Safety

“…Thereby, for safety reasons, the utility functions can and should include context-sensitive and perceiver-dependent elements as integrated e.g. in augmented utilitarianism [13]. Fourth, updates of law are solely reflected in the ethical goal functions which leads to a more flexible and controllable task.…”

“…Fifth, a blockchain approach to ensure the security and transparency of the goal functions themselves and all updates on these functions might be recommendable. Crucially, in order to avoid formulations of an ethical goal function with safety-critical side effects for human entities (including implications related to impossibility theorems for consequentialist frameworks [152]), it is recommendable to assign a type of perceiverdependent and context-sensitive utility to simulations of situations instead of only to the future outcome of actions [14,13]. In the long-term, we believe that scientific research with the goal to integrate the first-person perspective of society on perceived well-being within an ethical goal function at the core of the presented socio-technological feedbackloop might represent one substantial element needed to promote human flourishing in the most efficient possible way aided by the problem solving ability of AI.…”

“…From an abstract point of view, one could distinguish different means by which an adversary might achieve successful attacks: e.g. 1) by fooling the AI at the perception-level (in analogy to classical adversarial examples [198], this variant has been denoted ethical adversarial examples [13]) which could lead to an unethical behavior even if the utility function would have been aligned with human ethical Figure 6.1: Intuitive illustration for the law of requisite variety. Taken from [320].…”

“…states (of affairs) s . In line with this insight, is the context-sensitive and perceiverdependent type of utility functions considering agent, action and outcome which has been recently proposed within a novel ethical framework denoted augmented utilitarianism [13] (abbreviated with AU in the following) which was introduced in Chapter 4. Reconsidering the dyadic morality template iA d − → vP , it seems that in order to better capture the variety of human morality, utility functions -now transferring it to the perspective of Type I AI systems -would need to be at least formulated at the abstraction level of a perceiverdependent evaluation of a transition s a − → s leading from a state s to a state s via an action a.…”

“…For the purpose of facilitating the injection of requisite knowledge reflecting the variety of human morality in ethical goal functions, Section 6.2 provides information on the following variety-relevant aspects: 1) the essential role of affect and emotion in moral judgements from a modern constructionist neuroscience and cognitive science perspective followed by 2) dyadic morality as a recent psychological theory on the nature of cognitive templates for moral judgements. In Section 6.3, we propose first guidelines on how to approximately formulate ethical goal functions using a recently proposed non-normative socio-technological ethical framework grounded in science called augmented utilitarianism [13] that might be useful to better incorporate the requisite variety of human ethical intuitions (especially in comparison to classical utilitarianism). Thereafter, we propose how to possibly validate these functions within a socio-technological feedback-loop [14].…”